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Abstract 



Several classes of irreducible orthogonal representations of compact Lie groups that are of 
importance in Differential Geometry have the property that the second osculating space of all 
of their nontrivial orbits coincide with the representation space. We say that representations 
with this property are of class O^. Our approach in the present paper will be to find 
restrictions on the class (9^ and then apply them to classify variationally complete and 
taut representations. The known classifications of cohomogeneity one and two orthogonal 
representations and more generally of polar representations will also follow easily. 
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1 Introduction 



Several classes of irreducible orthogonal representations of compact Lie groups that are of 
importance in Differential Geometry have the property that the second osculating space of all 
of their nontrivial orbits coincide with the representation space. We say that representations 
with this property are of class O^. Our approach in the present paper will be to find 
restrictions on the class and then apply them to classify variationally complete and 
taut representations. The known classifications of cohomogeneity one and two orthogonal 
representations and more generally of polar representations will also follow easily. We refer 
to Section |^ for the definitions of concepts used in this introduction. 

Variationally complete actions were introduced by Bott in Bott and Samelson proved 
in that isotropy representations of symmetric spaces are variationally complete. We will 
prove the following converse of their theorem in Section ^. 

1.1 Theorem A variationally complete representation of a compact connected Lie group is 
orbit equivalent to the isotropy representation of a symmetric space. 

We will in fact prove more than we have stated here and give a complete classifica- 
tion of variationally complete representations (up to image equivalence; see the beginning 
of Section ^ for this concept). As a corollary we get a new proof of the classification of 
polar representations due to Dadok ([|T2|) since these are variationally complete (see 0) and 
isotropy representations of symmetric spaces are easily seen to be polar. It follows that an 
orthogonal representation is variationally complete if and only if it is polar, and it is polar if 
and only if it is orbit equivalent to the isotropy representation of a symmetric space. 

Another important result of Bott and Samelson in |^ is that the distance functions to 
orbits of variationally complete representations are perfect Morse functions. We say that 
submanifolds with this property are taut and call representations taut if all of their orbits 
are taut. In Section |^ we will prove the following classification theorem which provides new 
examples of taut submanifolds and taut representations. 

1.2 Theorem A taut irreducible representation p of a compact connected Lie group G is 
either orbit equivalent to the isotropy representation of a symmetric space or it is one of the 
following orthogonal representations (n > 2): 



G 


P 


SO(2) X Spin(9) 
U(2) X Sp(n) 
SU(2) X Sp(n) 


(standard) ^B. (spin) 
(standard) (standard) 
(standard)^ (8>h (standard) 



As a consequence of this theorem, the classes of polar, variationally complete and taut irre- 
ducible representations of compact simple groups coincide. Notice that the three exceptional 
representations in Theorem |1.2| are precisely the irreducible representations of cohomogeneity 
three that are not polar (see [0). Hence, in the case of a general compact Lie group, the 



taut irreducible representations are precisely those that are either polar or of cohomogeneity 
three. A further related result is the classification of homogeneous taut submanifolds with 
flat normal bundle due to Olmos (P^). He shows that such a submanifold is a principal 
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orbit of the isotropy representation of a symmetric space. It follows that the principal orbits 
of the three nonpolar representations in Theorem [L^ do not have flat normal bundle which 
is also easy to see directly. 

Bott and Samelson proved the tautness of variationally complete representations as an 
application of their i^-cycles which are concrete representatives of a basis for the homology 
of the orbits. (In |^ the group acting is denoted by K which explains the terminology 
'i^'-cycle'). The three nonpolar representations in Theorem |1.2| admit generalizations of the 
i^'-cycles of Bott and Samelson. We will not verify this claim in the present paper since it 
requires methods quite different from those employed here and would make the paper too 
long; we refer instead to the forthcoming paper ||2^. Instead we will prove their tautness 



with shorter, although maybe less illuminating arguments based on a theorem of Floyd ( [|17|] ). 
This approach is adapted from the paper [|1^ of Duistermaat. 

Kuiper observed in ||2^ that the second osculating space of a taut submanifold in a 



Euclidean space V coincides with V if the submanifold is not contained in a proper afiine 
subspace. In fact, he proved this more generally for tight submanifolds, but this is unimpor- 
tant for us since an orbit is tight if and only if it is taut. Since the classes of representations 
we are dealing with are all taut, it follows from this observation of Kuiper that they belong 
to class if they are irreducible. The class is much more tractable than the other classes 
of representations we are dealing with since it involves an infinitesimal condition. The tech- 
nique of Dadok ([|l^), notably his invariant k{X), turns out to be an extremely powerful tool 



to reduce the class in size so that the remaining cases are accessible to the geometric 
methods developed in Section ||. 

We have already pointed out that Dadok's classification of polar representations follows 
as a consequence of Theorem Some other classification results can also easily be proved 
with our methods as shown in Section ^. These include the classification of cohomogeneity 
one representations as well as the classification of cohomogeneity two representations due 
to Hsiang-Lawson (||2^). The cohomogeneity two representations are polar and therefore 
included in Dadok's classification, but the point here is that it is very easy to see directly 
that they belong to class without referring to tautness. 

One could also use our methods to classify symmetric spaces and their isotropy represen- 
tations. This would not lead to a simpler proof than those existing in the literature. Insisting 
on the details needed to classify symmetric spaces and their isotropy representations would 
make the paper much longer and more involved than it is. We will therefore use this clas- 
sification in the papeiQ and assume knowledge of the lists in Tables 8.11.2 and 8.11.5. 



(See also p4|, but notice that the isotropy representations are not listed in this reference). 
Otherwise, our guiding principle in writing the paper has been to make it as self-contained 
as possible. 



Previous to this paper taut representations were studied in and [[1^. It is decided 
in [0, with some exceptions, which representations of SU(n) and U(n) can be taut. In |Tl| 
it is proved among other things that a compact group admitting an almost faithful taut 
representation can have at most four simple factors. Eschenburg and Heintze use the theory 
of isoparametric submanifolds in |]l6l to reprove Dadok's characterization up to orbit equiva- 



lence of polar representations under the assumption of cohomogeneity at least three. In |T3 
^Except for the classification of cohomogeneity one representations. 
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they use the same theory to decide which irreducible representations are orbit equivalent 
to isotropy representations of symmetric spaces. This follows immediately from our work 
in Section ^. Bergmann in [|1| decides which reducible representations are polar. Her result 
is equivalent to the last step in our classification of variationally complete representations. 
Di Scala and Olmos give in a direct proof of the fact that a variationally complete 
representation is polar. 

The first author wishes to thank the Alexander von Humboldt Foundation for its generous 
support and constant assistance during the completion of this work. 



2 A review of basic definitions and results 

Let G be a compact Lie group acting on a Riemannian manifold M by isometrics. A geodesic 
7 in M is called G-transversal if it is orthogonal to the G-orbit through 'y{t) for every t. One 
can show that a geodesic 7 is G-transversal if there is a point to such that 7(^0) is orthogonal 
to G'jlto). A Jacobi field along a geodesic in M is called G-transversal if it is the variational 
vector field of a variation through G-transversal geodesies. The action of G on M is called 
variationally complete if every G-transversal Jacobi field J in M that is tangent to the G- 
orbits at two different parameter values is the restriction of a Killing field on M induced by 
the G-action (see @] and [^]). It is proved in P] on p. 974 that instead of requiring tangency 
at two different points in the definition of variational completeness it is equivalent to require 
tangency at one point and vanishing at another point. 

Let p be a variationally complete reducible representation and p a summand of p. Then 



it is easy to see that p is also variationally complete, see Proposition |3.4 Notice though that 
it is not true that the direct sum of two variationally complete representations of a compact 
Lie group G is variationally complete as can be seen by taking the direct sum of of two copies 
of SO(2) acting on R^. 

Let be a properly embedded submanifold of a Euclidean space V and let x be some 
point in V. Then we define the distance function : N ^ H from x to N hj setting 
Lx{p) = \ \p ~ 2^1 It follows that Lx is a non-negative proper function since N is properly 
embedded. Hence it is possible to apply Morse theory to L^. We say that is perfect with 
respect to a field F if it is a Morse function and the Morse inequalities for with respect 
to F are equalities. Furthermore we say that N is F-taut or simply taut if is perfect 
with respect to F whenever is a Morse function, see M. The concept of tautness can be 



extended to submanifolds of complete Riemannian manifolds, see [^, but we will not need 
that here. We will say that an orthogonal representation p : G — » O(V^) of a compact Lie 
group G is F-taut or simply taut if the the orbits of G are F-taut submanifolds of V. 

If p is a reducible taut representation and p one of its factors, then p is clearly taut. 
Notice that it is not true that direct sums of taut representations of a compact Lie group G 
are also taut. An example of this is the direct sum of n copies of SU(n) acting on C". This 
representation is not taut since one can compute that SU(?t,) is not taut in the matrix space 
M{n;C). 

The following theorem mentioned in the introduction was proved by Bott and Samelson 
more generally for variationally complete actions on complete Riemannian manifolds. In fact 
it is an immediate corollary of Theorem I in [O . Notice that Bott and Samelson do not use 
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the concept of a taut submanifold which was introduced later. 



2.1 Theorem (Bott-Samelson) A variationally complete representation of a compact con- 
nected Lie group on an Euclidean space is Z2-taut. 

Bott proved in [Q, Propositions 7.1 and 11.6, that the action of a compact Lie group 
G with a bi-invariant metric on itself by conjugations and the adjoint representation of G 
on its Lie algebra g are variationally complete. This was generalized by Bott and Samelson 
in P , Theorem II as follows. Let (L, G) be a symmetric pair and [ = g © p the corresponding 
Cartan decomposition. Then the action of L x L on L, the action of G on L/G and the 
action of AdiiG) restricted to p are variationally complete. The action of AdiiG) on p is 
equivalent to the isotropy representation of G on the tangent space Tp{L/G) where p denotes 
the coset G. Therefore we have the following theorem mentioned in the introduction. 

2.2 Theorem (Bott-Samelson) The isotropy representation of a symmetric space is vari- 
ationally complete. 

Conlon considered in actions of a Lie group G on a complete Riemannian manifold M 
with the property that there is a connected submanifold S of M that meets all orbits of G in 
such a way that the intersections between S and the orbits of G are all orthogonal. Such a 
submanifold is called a section and an action admitting a section is now usually called polar 
if S is properly embedded. Notice that Conlon does not assume in that S is properly 
embedded, but it is usually required in the recent literature on the subject. It is easy to 
see that a section E is totally geodesic in M. An action admitting a section that is flat 
in the induced metric is called hyperpolar. There is clearly no difference between polar and 
hyperpolar representations since totally geodesic submanifolds of a Euclidean space are affine 
subspaces. Moreover, the question whether S should be required to be properly embedded 
or not becomes redundant. Conlon proved the following theorem in 0. 

2.3 Theorem (Conlon) A hyperpolar action of a compact Lie group on a complete Rie- 
mannian manifold is variationally complete. 



Polar representations were classified by Dadok in . We recall that two representations 
Pi : Gi — s> 0{Vi) and P2 '■ G2 ^ 0(V2) are said to be orbit equivalent if there is an isometry 
A : Vi ^ V2 under which the orbits of Gi and G2 correspond. As a consequence of his 
classification he obtained the following result. 

2.4 Theorem (Dadok) A polar representation of a compact connected Lie group is orbit 
equivalent to the isotropy representation of a symmetric space. 

We can summarize this discussion as follows. Let G be a compact connected Lie group. 
Denote by X the representations of G that are isotropy representations of symmetric spaces, 
by V those that are polar, by V those that are variationally complete, and by T those that 
are taut. Then 

ICPCVCT. 

The starting point in the present paper will be to consider a class of representations that 



is a priori larger than T, but easier to deal with, see [11 1. We recall that the second osculating 



7 



space Op{N) at a point p of a submanifold in a Euclidean space V is the vector space 
spanned by the first and second derivatives at p of the inclusion of into V. It is easy to 
see that 

OliN) = TpN © Y)\X,Ye T,N}). 

where a denotes the second fundamental form of and (S) stands for the linear hull of the 
set S. The following is a corollary of the discussion of Kuiper in P? . 



2.5 Theorem (Kuiper) Let N be a taut submanifold of a Euclidean space. Then the affine 
hull of N coincides with p + 0^{N) for every point p in N . 

We let denote the class of representations of a compact connected Lie group G such 
that the representation space coincides with 0^{Gp) for all nonzero p. The representations 
in are clearly irreducible. If a taut representation is irreducible, then Theorem |2.5| implies 
that it belongs to class C^. Thus we have the inclusions 

lidViGVidTid C^, 

where Xj, Pj, Vi, and % are the the subclasses of X, P, V, and T consisting of irreducible 
representations. 



3 A collection of geometric propositions 

In this section we collect technical results of geometric nature that will be needed to prove 
our main theorems. 

3.1 Variationally complete representations 

We first study focal points of orbits of variationally complete representations with the aim 
to prove that a representation that is orbit equivalent to a variationally complete one is also 
variationally complete. We then show how reducible variationally complete representations 
relate to irreducible ones. Finally we prove an important criterion which can be used to 
decide in some cases whether a variationally complete representation is orbit equivalent to 
the isotropy representation of a symmetric space. 

Let g be a focal point of an orbit Gp of an orthogonal representation of a compact 
connected Lie group G. It is clear that the orbit Gq consists of focal points of Gp. We 
therefore call Gq a focal orbit ofGp. It is also clear that the singular orbits of a representation 
are always focal orbits of the principal orbits. Slightly more generally, Gq is a focal orbit of 
Gp if the segment qp is perpendicular to Gp in p and there is a one-parameter group in Gq 
that does not fix p. We will use the following lemma to prove Proposition |3.3| . 

3.1 Lemma A representation p : G ^ O(^) ^■5 variationally complete if and only if every 
orbit Gp has the following property: if q is a focal point of Gp relative to p with multiplic- 
ity k > 0, then k = dimGqp. 
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3.2 Remark (Compare ^2[.) Let p : G ^ O(^) be a variationally complete representation 
of a compact connected Lie group G. Then Lemma |3.1| implies that the focal orbits of 
any orbit of G are singular orbits. Furthermore, the multiplicity of the focal points of a 
principal orbit Gp in a singular orbit Gq is equal to the difference between the dimension of 
Gp and the dimension of Gq. In particular, it follows that any principal orbit is a totally focal 



submanifold; see [42] for a definition of this concept. Terng claims in that this fact can 
be used to show that variationally complete representations are polar. However, as Terng 
herself pointed out in ^3|, p. 197, there is a gap in that argument. 



Proof of Lemma \3. 1\ . We first assume that p is variationally complete. Let g be a focal 
point of Gp along the line segment pq with multiplicity k. Let be the space of Jacobi fields 
along pq that are the variational vector fields of variations 7^ of pq through line segments such 
that all 7s meet the orbit Gp orthogonally for t = and the Jacobi field J of the variation of 
7 vanishes for t = 1. The multiplicity k of the focal point q is equal to the dimension of the 
space The action of Gq on pq induces variations of pq whose variational vector fields are 
contained in J'. Let us denote by J' the subspace of Jj that consists of Jacobi fields induced 
by the action of Gq on pq. It follows that dimjT" = dim GqV < k where v is the tangent 
vector of pq at q. We now prove that dim GqV = k. Let 7^ be a variation of pq through 
line segments such that all 7^ meet the orbit Gp orthogonally for t = and the Jacobi field 
J of the variation along 7 vanishes for t = 1. The segments 7^ are orthogonal to the orbit 
Gp. Hence they are G-transversal and J is a G-transversal Jacobi field. It is clear that J is 
tangent to the orbits through p = 7(0) and q = 7(1). By variational completeness, there is 
a Killing field X on M induced by the G-action such that J = X o 7. Hence J ^ J and it 
follows that J = J . Thus dimG^f = k. 

To prove the converse, we have to show that dimG^f = k for all focal points q implies 
that p is variationally complete. This is equivalent to show that a Jacobi field J that vanishes 
in a point q and is tangent to an orbit Gp in p is induced by the group action. We let the 
symbols J and J have the same meaning as in the first part of the proof. We have to show 
that J = J . This is clear since dam J = k and dim = dim GqV = k. □ 

The following proposition that we will use to classify variationally complete representa- 
tions is now easy. 

3.3 Proposition Let p be a representation that is orbit equivalent to a variationally complete 
representation. Then p is itself variationally complete. 

Proof. We can assume that we have representations p and p of the groups G and G 
with the same representation space and the same orbits and that p is variationally complete. 
Clearly the focal orbits of the two representations coincide since their orbits are the same. 
Let f be a normal vector in q of the orbit Gq = Gq. It follows from the slice theorem and the 
fact that orbits of p and p are the same that GgV = GqV. Since p is variationally complete. 
Lemma now implies that p is also variationally complete. □ 



The next proposition will be important in the proof of Theorem |1.1| in the introduction. 



It is analogous to a result of Dadok on polar representations, see Theorem 4 in 
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3.4 Proposition Let G be a compact connected Lie group. Let p : G ^ O(^) be a non- 
trivial reducible variationally complete representation and let p = pi ® P2 be a nontrivial 
decomposition. Let Vi and V2 be the representation spaces of pi and p2 respectively. Then 
we have the following: 

(a) The representations pi and p2 are variationally complete. 

(b ) The restriction of p2 to the connected component of the isotropy subgroup of any point 
in Vi is orbit equivalent to p2. 



'ions. 



(c) The representations pi and p2 are not equivalent as representat 

(d) The representation p is orbit equivalent to the representation p : GxG ^ O(^) defined 
by p{9i,92) = Piigi) +P2{92)- 

Proof. The claim in part (a) is clear, since the orbits of pi and p2 are also orbits of p. 

To prove part (b), note that it is true in general, without assuming p is variationally 
complete, that any point pi G Vi is a focal point of any orbit Gp2 C V2 with multiplicity 
equal to dimGp2- Now if p is variationally complete then the claim follows from Lemma |3]^ 
and the fact that the G-orbits are connected. 

Next we prove (c). Assume A : \4 — ^ V2 is an isomorphism such that p2 = ApiA~^. 
Notice that Gp = Gap for all p & Vi. Part (b) implies that Gp is transitive on the orbit 
G{Ap)., which is impossible if we choose p G Vi such that dimGp > 0. Therefore such an 
isomorphism A does not exist. 

Finally we prove part (d). Let f = f 1 + ^2 be a point in V and Vi and V2 its components in 
Vi and V2 respectively. Let Hi (resp. H2) be the isotropy group of Vi (resp. ^2). Notice that 
V2 is a focal point of Gvi with multiplicity dirnGvi. It follows from part (b) that H2V1 = Gvi 
and similarly that H1V2 = Gv2. We next prove the factorization G = H1H2. Let g & G. 
Then there is an element hi G Hi such that hiV2 = gv2- Then hi^g G H2. Set /i2 = hi^g. 
Then g = hih2. It follows that 

9 = i)i + i)2 

where g, f)i and ()2 are the Lie algebras of G, Hi and H2 respectively. It is clear that the orbit 
of V under p{G) is contained in its orbit under p{G x G). The tangent space of p{G x G)v 
at V is 

(0 X g)v = gvi + gv2 = i)2Vi + i)iV2. 
Let Xi G f)i and X2 G f)2- Then XiVi = and X2f2 = and hence 

X1V2 + X2V1 = (Xi + X2)v G gv. 

Hence the dimension of the tangent spaces of the orbits of v under p and p coincide. Since G 
is connected the orbits coincide. Since v was chosen arbitrarily, all orbits p and p coincide. 

□ 



3.5 Remark Part (c) in Proposition |3.4| generalizes Proposition 2.10 in |23| . 
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The next lemma is a preparation for Proposition T7_. We follow the terminology of Bredon 
in 1^, pp. 180-181, and distinguish between three types of orbits, namely principal (regular), 
exceptional and singular. Recall that orbits are called exceptional if they have maximal 
dimension without being principal, and singular if their dimension is not maximal. We also 
call points regular when they belong to principal orbits. 

3.6 Lemma Let G be a closed subgroup of the compact connected Lie group K and let 
p : K ^ 0(y) be a representation. If p has an orbit of maximal dimension in common with 
p\G, then p and p\G are orbit equivalent. 

Proof. We denote the maximal dimension of an orbit of p by n. Let us denote by Og 
the union of the G-orbits of maximal dimension and by Ok the corresponding set relative 
to K. It is well-known that both Oq and Ok contain an open and dense subset of V . Hence 
the set Og H Ok also contains an open and dense set. The dimensions of the orbits of G 
and K through points in Og H Ok are all equal to n. It follows that the orbits of G and K 
through points in Og H Ok coincide since K is connected. Let p &V \ Og H Ok- We would 
like to show that Gp = Kp. Let k{p) be some other point in Kp. Let {pn) be a sequence 
in Og H Ok that converges to p. Then {k{pn)) converges to k{p). There is for every n an 
element gn in G such that gnipn) = k{pn). Now {gn{Pn)) converges to k{p). By going to a 
subsequence if necessary we can assume that (?„ converges to an element g in G, so that the 
continuity of G x V ^ V implies that [gniPn)) converges to g{p). Hence g{p) = k{p) and we 
have seen that all orbits of G and K coincide. □ 

3.7 Proposition Let G be a closed subgroup of the compact connected Lie group K and 
let p : K —>■ 0{V) be the isotropy representation of an irreducible symmetric space. If the 
restriction p\G is variationally complete and p\G has a nontrivial orbit in common with p, 
then p and p\G are orbit equivalent. 

Proof. Let p & V he a point such that the orbits of G and K through p coincide and 
have the largest possible dimension with this property. Assume that Kp is not an orbit of 
maximal dimension for the action of K. Let a be a Cartan subspace in V with respect to 
the symmetric space representation of K such that p G a. Let T be a tubular neighborhood 
of the common orbit Kp = Gp. Let g G a be a point in Gp = Kp different from p and such 
that the segment pq from p to q passes through a point r in T with dim i^^r > 1. (Here it is 
important that the symmetric space is irreducible.) It follows that dim Kp < dim Kr by the 
slice theorem. Hence dim Gr < dim Kr since the dimension of Gp = Kp is maximal among 
the common orbits of G and K. It follows that dim Gpr < dim KpV. 

Let X be an element of the Lie algebra tp of Kp such that 7x.r(0) is not in Tr{Gpr) where 
lx,r{s) = exp(sX)r. Now let us consider the variation (pg)s = exp(sX)(pg) of pq. Let J be 
the corresponding Jacobi field along pq. This variation is fT-transversal and hence also G- 
transversal. The value of J over p and q are tangent vectors of the common orbit Gp = Kp. 
By the variational completeness of the action of G, the Jacobi field J is the restriction of 
a Killing field on V to pq that is induced by the action of G on V. Hence all values of J 
are tangent vectors to orbits of G. The value of J over r is 7x,r(0) which is not a tangent 
vector of the orbit of G through r. This is a contradiction. Hence G and K have an orbit in 
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common which is of maximal dimension among the i^-orbits. It now follows from Lemma 3^ 
that all orbits of G and K coincide. □ 



3.2 Taut representations 

We will first prove that the slice representation of a taut representation is also taut and 
end the section with an important proposition about taut reducible representations. It also 
follows immediately from our methods that a taut representation does not have exceptional 
orbits. 

3.8 Proposition Let G he a compact connected Lie group with a taut representation p : 
G — >■ 0{V). Let p E V and let Up : Gp 0{Np{Gp)) be the slice representation of p at p. 
Then Vp is taut. 

Proof. Let N'^iGp) denote the bundle of normal vectors of length less than e over the 
orbit Gp. Let e > be so small that T = exp(A^'^(Gp)) is a tubular neighborhood around the 
orbit Gp. Let v G Np{Gp). We want to show that GpV is taut in the normal space Np{Gp). 
Let a > be a number that is so small that w = av has length less than e. It is clear that 
GpV is taut in Np{Gp) if and only if GpW is taut in Np{Gp). It is also clear that GpW is taut 
in Np{Gp) if and only if expp(Gpw) = Gpexpp(w) is taut in expp(iVp(Gp)) since exp^ is an 
isometry. We set q = expp{w). A submanifold in an affine subspace A of K is taut in A if 
and only if it is taut in V. Hence GpV is taut in Np{Gp) if and only if Gpq is taut in V . 

Now let Lp : Gq ^ R be the distance function. The segment pq is orthogonal to Gp 
and hence also to Gq. It follows that g is a critical point of Lp. Since q is in the tubular 
neighborhood T and the length of the segment pq is smaller than e it follows that Lp takes on 
its minimum in q. Let G be the subset of Gq on which Lp takes on its minimum value. Clearly 
G = Gq n expp{Np{Gp)) since a geodesic segment between Gp and Gq that is orthogonal 
to Gp is orthogonal to Gq and vice versa. By the slice theorem Gpq = Gq H expp{Np{Gp)). 
Hence Gpq = G. The set of critical points of a distance function to a taut submanifold is a 
union over taut submanifolds by a theorem of Ozawa, see P3|. It follows that Gpq is taut 



and hence that the slice representation i/„ is taut. □ 



3.9 Remark We are assuming in Proposition |3.8| that G is connected. Hence its orbits are 



connected and it follows from their tautness that the set of points where a distance function 



takes on its minimal value is connected. The proof of Proposition 3.8 now implies that the 



orbits of Gp are connected even if Gp is not connected. 

It is well-known that the isotropy representation of a symmetric space does not have an 
exceptional orbit. The methods used in the proof of Proposition |3.8| immediately give that 
this is also the case for taut representations, as we show in the next proposition. 

3.10 Proposition A taut representation p : G ^ O(^) ^f compact connected Lie group 
G does not have exceptional orbits. 

Proof. Assume the orbit through p in is exceptional. Then the slice representation of 
Gp at p has a disconnected orbit. Arguing exactly as in the proof of Proposition BTBI, we see 



that the distance function from p to a principal orbit through some regular point close to 
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p assumes its minimum value on a disconnected set, contradicting tautness. Thus p cannot 
have an exceptional orbit. □ 

We close this subsection with a discussion of taut reducible representations. 

3.11 Proposition Let pi and p2 be representations of a compact connected Lie group G 
with representation spaces Vi and V2 respectively. Assume that pi © p2 is F-taut. Then the 
restriction of p2 to the isotropy group G^^ is taut for every f 1 G Vi . 

Furthermore, we have p{G{vi,V2); F) = p{Gvi; F) p{GviV2; F) where p{M; F) denotes the 
Poincare polynomial of M with respect to the field F . In particular, Gy^V2 is connected and 

bi{G{vi, V2y,F) = bi{Gvi; F) + bi{G,,V2; F) 

where &i(M; F) denotes the first Betti number of M with respect to F. 

Proof. We can work with height functions instead of distance functions in this proof since 
a submanifold contained in a round sphere in a Euclidean space is taut if and only if all 



height functions are perfect Morse functions, see ^5[|. Furthermore, in this situation the set 
of critical points of a distance function will also occur as the set of critical points of a height 
function, and vice versa. Fix (fi,f2) G Vi © V2. Let a E Vi he such that the height function 
ha : Gvi — > R defined by ha{v) = {a,v) is a Morse function. We define the height function 
h{a,o) '■ G{vi,V2) — > R similarly. The point (mi,M2) is a critical point of /i(a,o) with index i if 
and only if Ui is a critical point of ha with index i. Hence the critical set C on the critical 
level h(a,o){ui,U2) = ha{ui) is 

C = {{wi, W2) e G{vi, V2) I wi = ui} = {{ui, W2) I W2 e Gu^v2}. 



Ozawa proves in ||3J] that the set of critical points of a distance function on a taut submanifold 
is a union over nondegenerate critical submanifolds that are again taut submanifolds. It 
follows that C is taut. The projection of C into V2, which coincides with Gu^V2, is then also 
taut. We can choose a such that ha is a Morse function of which vi is a critical point. It 
follows that Gv^V2 is taut for every vi G Vi and V2 G V2 and hence that the restriction of p2 
to Gv-^ is taut. 

Now fix again a point (fi,f2). If C C ^(f 1,172) is an arbitrary critical submanifold of 
/i(a,o), then it is diffeomorphic to Gy^V2. To see this let vri : G{yi, V2) Gvi be the projection 
onto the first factor. Then one easily sees that 

T^i^igvi) = Ggy^gv2 = gG.,,g'^{gv2) = gGy^V2. 

Hence we have that tti is a G-equivariant fibration. The critical submanifolds of /i{a,o) are 
fibers of tti and hence diffeomorphic to each other. 

Since the orbit G{vi,V2) is taut we have by that the Morse-Bott inequalities are 



equalities. Let Ci, . . . , be the critical manifolds of /i(a,o)- Let i{Cj) be the index of Cj for 
j = 1, . . . , k. Then 



p{G{v„V2)) = J2piCj)f^''^^ = piGy-,V2)J2^ 
j=i j=i 



iiCj) 
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since the critical submanifolds are all diffeomorphic to Gy^V2- We have 

k 

since ha is a perfect Morse functions with critical points pi = 7ri(C), . . . ,Pk = 7ri(Cfc) and 
the index of pj is equal to i{Cj). It follows that 

piG{vi,V2)) = p{Gvi)p{G^^V2). 

Multiplying out the Poincare polynomials gives hQ{G^^V2) = 1 and 

h{G{vi, V2)) = h{Gv,) + h{G,,V2). 

This finishes the proof of the proposition. □ 



3.12 Remark The property that p2|G'i)i is taut in Proposition p.ll is a weakening of the 



property that p2\Gy-^ is orbit equivalent to p2 for a variationally complete reducible represen- 
tation pi © P2 (cf. Proposition |3^ , part (b)). 

We give examples of how Proposition p.ll| can be used. 

3.13 Examples (i) Let G = SO(n) and let pi be the SO(?2)-conjugation on the space Vi of 
real traceless symmetric nxn matrices. Then pi is taut since it is the isotropy representation 
of the symmetric space SL(n, R)/SO(n). Let p2 be any other nontrivial representation of 
SO(n) with representation space V2. Then pi © p2 cannot be taut if n > 3. To see this, let 
vi G Vi be a regular point. Then G^^ is the discrete group consisting of all diagonal matrices 
with determinant one and entries ±1 on the diagonal. The kernel of p2 is contained in the 
center of SO(n). Since > 3, we see that G^^ cannot be contained in the center of p2. Hence 
there is an element ^2 € V2 that is not fixed by G^-^. It follows that Gjj-^V2 is disconnected. 
Now Proposition ^.11| implies that pi © p2 is not taut. 



(ii) Now let G be a compact connected simple Lie group of rank at least two and let 
pi denote the adjoint representation of G. We assume that G is simply connected. Let p2 
be any other nontrivial representation of G. Then pi © p2 is not taut. To see this let T 
be a maximal torus in G. We denote the representation spaces of pi and p2 by Vi and V2 
respectively. There is a regular element Vi G Vi with = T. The restriction of p2 to T has 
a discrete kernel that is contained in the center of G. If V2 G V2 is a T-regular point then 
the isotropy subgroup coincides with the kernel of P2IT. Hence Gy-^V2 is diffeomorphic to 
T and it follows that 6i(G^if2; F) is equal to the rank of G. In particular bi{Gv^V2] F) > 2. 
Now notice that the isotropy group of (^1,^2) is also T^^. Hence vri(G(fi, ^2)) = T^^ which 
implies Hi{G{vi, V2); Z) = T^^ since T^^ is Abelian. If G 7^ Spin(4/c) then the center of G is a 
cychc group and it follows that 6i(G(fi, ^2); F) < 1. If G = Spin(4A;), then k >2 and we get 
6i(G^^f2; F) = 2k > 4; since the center of Spin(4A;) is Z2 x Z2, we have bi{G{vi,V2); F) < 2. 
In either case, 6i(G„jf2; F) > &i(G(f 1, f2); F) which implies by Proposition |3.11| that pi © p2 
is not taut. 
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3.3 A reduction principle 

We now discuss a reduction principle which will be used later to prove that certain rep- 
resentations are or are not taut. Let p : G ^ O(^) be an orthogonal linear action of a 
compact Lie group G which is not assumed to be connected. Denote by if a fixed principal 
isotropy subgroup of the G-action on V and let be the subspace of V that is left point- 
wise fixed by the action of H. Let N be the normalizer of H in G. Then the group N/H 
acts on with trivial principal isotropy subgroup. Moreover, the following result is known 
(0,^1, |2|, 0,111,11): 



3.14 Theorem (Luna-Richardson) The inclusion — > V induces a stratification pre- 
serving homeomorphism between orbit spaces 



(3.15) V"/N ^ V/G. 



The injectivity of the map ( p.l5|) means that Np = Gp fl for p G . In particular. 



the H-fixed point set of a G-orbit is a smooth manifold. Observe also that for a regular point 
p G the normal space to the principal orbit M = Gp at p is contained in , because 
the slice representation at p is trivial. 



3.16 Lemma Let p G be a regular point and consider a normal vector ^ to M = Gp 
at p. Then the Weingarten operator of M at ^ restricts to the Weingarten operator of 

\M\ — aM " 



= M nV" at ^, m symbols, |r = 



Proof. Let v G TpM^ . Consider an extension ^(t) of ^ to a normal vector field along a 
curve 7(t) in with 7(0) = v. Then A^'^v = ~^'{0)^, where denotes the orthogonal 
projection onto TpM. Now ^{t) G , so the derivative ^'(0) G . The normal component 
of f'(0) is already in , so its component in TpM is also in . Hence A^^"v = —^'{0)^ 
which proves the claim. □ 

3.17 Lemma Let p, q E and suppose that q is a regular point for G. Consider M = Gp 
and let Lg : M —>■ H be the distance function. Then the critical set of Lg is contained in 
= Mn V" , namely, Crzt(Lg) = Crzt(Lq\MH ). 

Proof. Let x G M be a critical point of Lg : M R. Then x = gp for some g & G 
and V = q — X is a. normal vector to M at x. Consider first the case where p is a regular 
point for G. Then the isotropy subgroup at x is principal, so G^ = gHg~^ acts trivially 
on the normal space. In particular gHg~^v = v, which implies that gHg~^q = q. Now 
gHg~^ G Gq = H and therefore g G N. Thus x = gp G and x is a critical point for 
the restriction Lq : —>■ R. Consider now the case where p is not a regular point for G. 
Then there is e > sufficiently small such that y = x + ev is a. regular point for G. Now y 
is a critical point for Lq : Gy — > R, since a line segment meeting one orbit orthogonally is 
orthogonal to all orbits it meets. By the previous case y G . Hence x G and again x 
is a critical point for the restriction Lq : R. 

On the other hand, let x G be a critical point of Lq : —* R. Consider first 
the case where p is a regular point for G. Then x is also a regular point for G so that 
the normal spaces Nx{M^) and N^^M) coincide. It follows that x is a critical point of 
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Lq : M ^ H. Consider now the case where p is not a regular point for G. Then there is 
e > sufficiently small such that y = x + ev & is a regular point for G, where v = q — x. 
Since Gy fl = Ny, we get that y is a critical point for Lg : Ny — >• R. By the previous 
case y is a critical point for Lg : Gy —>■ R. Hence again x is a critical point of Lg : M — > R. 

□ 

For the next lemma and proposition we found inspiration in [14]. 

3.18 Lemma Suppose there is a subgroup L C H which is a finitely iterated Z2-extension 
of the identity and such that the fixed point sets = . Let M = Gp be a principal orbit. 
Then the sum of the Betti numbers ( with respect to Z2 coefficients ) of is less than or 
equal the sum of the Betti numbers of M, in symbols (3{M^; Z2) < /3(M; Z2). 

Proof. We can write L = {hi, . . . , hn} G H such that /ij+i normalizes and hf_^_i G 
for i : 0, . . . , n — 1, where Hq is the identity group and Hi is the subgroup of H generated by 
hi, . . . ,hi. Now each transformation /ij+i has order 2 in the fixed point set M^' = M fl V^', 
so by a theorem of Floyd ( ) 

/3(M^; Z2) < /?(M^"-^ Z2) < . . . < /?(M^0; Z2). 

But = and M^" = M and this completes the proof. □ 

3.19 Proposition Suppose there is a subgroup L G H which is a finitely iterated Z2- 
extension of the identity and such that the fixed point sets = . Suppose also that 
the reduced representation p : {N/H)^ 0{V^) is Z2-taut where [N/H)^ denotes the con- 
nected component of the identity of N/H. Then p : G —>■ 0{V) is Z2-taut. 

Proof. Let M = Gp be an arbitrary orbit. We shall prove that M is Z2-taut. Since 
intersects all orbits, we may assume p G . Suppose : M — > R is a Morse distance 
function. We must show that Lg is Z2-perfect, and in fact it is enough to do that for g in a 
dense subset of V, so we may take g to be a regular point. Since G acts on V by isometries, 
we may moreover assume that q G . If p is Z2-taut, then each connected component of 
is Z2-taut, hence itself is Z2-taut. Note that Lg\]^.jH is a Morse function. Hence 



Lg\]yjH is Z2-perfect. Now Lemmas |3.17| and p.l8| and the Morse inequalities imply that 



/3(M^; Z2) < /?(M; Z2) < #Crit(L,) = #Crit(L,|MH), 
where denotes the cardinality of the finite set A. It follows that Lg is Z2-perfect. □ 



4 A necessary condition for a representation to be of 
class 

Let 77 be a complex representation of the compact connected Lie group G on a finite- 
dimensional vector space. We say that vr is of real type if it comes from a representation 
of G on a real vector space by extension of scalars, and we say that n is of quaternionic type 
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if it comes from a representation of G on a quaternionic vector space by restriction of scalars. 
If vr is neither of real type nor of quaternionic type, we say that vr is of complex typef^ 

Now it is known that the finite-dimensional real irreducible representations p of G fall 
into one of the following disjoint classes: 

(a) the complexification is irreducible and p'^ = vr is a complex representation of real 
type; 

(b) the complexification p'^ is reducible and p*^ = vr © vr where vr is a complex irreducible 
representation of quaternionic type; 

(c) the complexification p'^ is reducible and p"^ = tt © tt* where vr is a complex irreducible 
representation of complex type and vr* is not equivalent to tt (where vr* denotes the 
dual representation of vr). 

The relation between p and vr is that p is a real form of vr in the first case (p*^ = vr), but p 
is TT viewed as a real representation in the other two cases (p = ir^). We shall call p of real, 
quaternionic or complex type according to whether the associated vr is of real, quaternionic 
or complex type. Note also that tt is self-dual precisely in the first two cases. 

Suppose now that G is semisimple, let q denote its Lie algebra and its complexification. 
Write A for the root system of g"^ with respect to a chosen Cartan subalgebra. A"*" for the 
positive root system with respect to an ordering of the roots, and S = {ai, . . . ,ar} for the 
corresponding simple root system. Let Ai, . . . , A^ be the fundamental highest weights defined 
by the relations 2(Aj, aj)/{aj, aj) = 6ij where ( , ) is the Cartan-Killing form. The Theorem 
of the Highest Weight of E. Cartan states that the complex irreducible representations of 
are parametrized by their highest weights, and these are exactly the linear combinations 
A = ^i^i'TJ'iAj for mi G {0,1,2,...}. The following proposition is a useful criterium of 
Dadok to decide for which A the corresponding representation tcx is of real, quaternionic or 
complex type. Recall that the roots a, /? e A are called strongly orthogonal if a ± /5 is not 
a root. 

4.1 Proposition ( ||12|| ) There is a maximal subset B = . . . , f3s} C A+ of strongly 
orthogonal roots such that: 

(a) we have = ■ ■ ■ s^^ is the Weyl group element that maps the positive Weyl chamber 
into its negative; 

(b) the representation tta is of complex type if and only if A does not belong to the real span 
ofB; 

(c) the representation tt^ is of real type (resp. quaternionic type) if and only if X belongs 
to the real span of B and 

k{X) = ^ t^' ~ 



1=1 



(A, A) 



^It is useful to remark that tt is of real type (resp., quaternionic type) if and only if there exists an 
invariant real (resp., quaternionic) structure on the representation space, i. e. a C-conjugate linear, G- 
invariant endomorphism e of the representation space such that = 1 (resp., = —1). 
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is an even (resp. odd) integer. 



4.2 Remark For a simple Lie algebra the set B can be constructed as follows, as is explained 
in 



| T^ and will be assumed throughout our paper (cf. Appendix A). Let Pi be the highest 
root. The root system {a G A : (a, Pi) = 0} is either irreducible or equals {±0} U Ai, with 
Ai being irreducible and (i G A+. In the former case set f32 equal to the highest root of Ai 
(with the inherited order from A), and proceed by induction. In the latter case set (32 = Ci 
and (3^ equal to the highest root of Ai, and proceed by induction. 

We continue to assume that G is semisimple. Let H^, a G A, be the coroots of g^. It 
is possible to choose root vectors Xa for g"^, a G A, such that the compact real form g is 
spanned by 

(4.3) iHa, Xa - X^a, i{Xa + ^-a), whcrc a G A+. 

Now if TT is a complex representation of G, we have that the adjoint map 7r(Xa)* = 7r(X_Q,) 
with respect to any G-invariant Hermitian product on the representation space. Moreover, if 
vr is of real (resp. quaternionic) type and e is an invariant real (resp. quaternionic) structure 
on the representation space, then TT{Xa)e = e7i{X^a)- 

By compactness of G, any real representation is equivalent to an orthogonal one. The 
following proposition, which is stated as a remark in |T2[, p. 128, and the ensuing lemma. 



which is a refinement of a result in |TT[, are precisely the ingredients we need to establish a 
necessary condition for an orthogonal representation of G to be of class O^. 

4.4 Proposition ( ||12|| ) Lethl^{Q'^) he the kth level in the natural filtration of the universal 
enveloping algebra of Q^. Fix ttx, the irreducible representation of q'^ with highest weight X, 
and fix v\, a highest weight vector. Let 

_ (A,A)-(goA,A) 

(A, A) ' 

/ori = l,... ,s. Then the element sqV\ is in IA^^^\q^)v\, but not inU^(g^)vx for any I < k(X), 
where sqVx is given as 

Sot;A = vr(X_^,r---7r(X_^J"»t;,. 



There is no proof of the above proposition in |0. In |]T0[, p. 270, there is an attempt to 
prove the proposition which in our view contains serious gaps. 

Proof. It is enough to consider the case where q'^ is a complex simple Lie algebra. We 
first claim that the element sqVx is not zero. In fact, consider the complex subalgebra i G q'^ 
generated by the root spaces of g'^ corresponding to ±Pi, . . . ,±Ps- Then t is isomorphic 
to the direct product of s copies of sl(2, C). We restrict vr^ to t and let Ux be the unique 
irreducible 6-module generated by vx- Since A — sqA = J2i=i^iPiy it is now clear that Ux 

is the representation space of 6 : o ® ■ ■ ■ ® o and that SqVx G Ux- Therefore, our claim is 
reduced to the case of the {n + l)-dimensional complex irreducible representation of s[(2, C), 

n 

namely o, which is immediate to verify. 
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It is obvious that SqVx G U^'^^\q'^)v\. We next prove that SqVx ^ U\q'^)vx for I < k{X) 
by contradiction. In fact, enumerate the positive roots A+ = {ai, . . . ,at} and suppose 
SqVx G U\q'^)vx for I < k{X). It follows from the Poincare-Birkhoff-Witt theorem that we 
can write 

pi,... ,pt>0 

for some complex constants Cp^.-.p^. Now, using that X* = 

7^ {sqVx, SqVx) 

= X] '^Pi ■ ■ -P' ("'^-"i ■ ■ ■ ^-»t v\,X'!_]^^ - ■ ■ X""^^ vx ) 

pi,... ,pt>0 
pi,... ,pt>0 

Recall that the highest weight space is one-dimensional; hence we can write 

X;:...X;iX^J^^...X^J^^Vx = cvx, 

for some nonnegative integers pi, . . . ,pt such that k{X) = rii + ■ ■ ■ + Ug > pi + ■ ■ ■ + pt = I 
and for some nonzero complex constant c. Note that riiPi + ■ ■ ■ + UgPs = piai + ■ ■ ■ + PtCtf 
The contradiction we are aiming at now follows from the following claim. 

Claim 1 If N > M then 

Xs,^ ■ ■ ■ Xsj^X_^^ ■ ■ ■ X_^j^jVx = 0, 
where 5i, . . . ,5n E B, and 71 , . . . , 7m G A+ . 

We proceed by induction on the integer N. The cases N = 1 and N = 2 are trivial (use 
that Vx is a highest weight vector for N = 1 and that the sum of two roots in B is not a 
root for N = 2). Assume the assertion is true for some N — 1 > 2 and let us prove it for N. 
Since , • • • , pairwise commute, we may assume that 61 = Pi where i < j for every 
j = 1, . . . ,s which satisfies Pj = 6k for some k = 1, . . . ,N. 

Claim 2 For each j = 1, . . . , M , if 61 — 7^- is a root then it is a positive root. 

Indeed we have 61 = Pi where i is as above. If i = 1, then we are done because Pi is the highest 
root. If not, = iPuSi + - ■ ■ + 6n) = (A, 7i + ■ ■ ■ + 7a/). Now (Pmi) = -(/^i, 72 + • ■ ■ + 7a/), 
and since Pi is the highest root we have (/3i,7i) > 0,... ,(/5i,7m) > 0. It follows that 
{Piyli) = and 72 H — ■ + 7a/) = 0. An easy induction argument shows that {Pi, 7^) = 
for j = 1,... , M. Consider now the root system {a G A : {a, Pi) = 0}. The first case 
occurs when it is irreducible. Then P2 is by definition its highest root so that P2 > 7j for 
j = 1, . . . , M. If i = 2 we are done. If not, we see that {P2, 7j) = for j = 1, . . . , M and 
we proceed by induction. The second case occurs when the above root system is of the form 
{±Ci} U Ai, with Ai being irreducible and (i G A"*". Here P2 = Ci ^"^^ Ps is the highest root 
of Ai. If 7j G Ai for j = 1, . . . , M then {P2, 7^) = for j = 1, . . . , M so that i > 3 and we 
proceed by induction as above. On the other hand, if (i = 7jQ for some Jq, then i = 2. In 
this case, for each j = 1, . . . , M either P2 = ■jj or — 7^- G Ai. So P2 — 7j is never a root. This 
completes the proof of Claim |. 
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Claim 3 We have 



where H,^ is the coroot vector corresponding to ( ^ and j = 1, . . . , M. 

In order to prove Claim ^, proceed by induction on j. The initial case is j = M, which 
follows from the induction hypothesis on A^, since H^^vx = {(, A)f a- Next write 

Xsff ■ ■ ■ Xs^X^^-^ ■ ■ ■ X_^^_^/f^X_^^^^ ■ ■ ■ X_^^^f A = 

(4.5) Xsj^ ■ ■ ■ Xs^X^^^ ■ ■ ■ ■ ■ ■ X^^j^^vx 

+ Xs^ ■ ■ ■ Xs^X_^^ ■ ■ ■ X_^^_^ [iJ^, X_^^,^J ■ • ■ X^^j^jVx- 

The first summand on the right hand side of ( [4 .51 ) is zero by the induction hypothesis on j, 
and the second summand is zero because [if^,X_^^.^J = —{(,'jj^i)X_^.^-^ so that we can use 
the induction hypothesis on N. This proves Claim ^. 

Claim 4 We have 

Xsn ■ ■ ■ Xs^X^^^ ■ ■ ■ X^^._-^X(^X^^.^-^ ■ ■ ■ X-^^jVx = 0, 
where is the root vector corresponding to Q E and j = 1, . . . , M. 

In order to prove Claim ^, proceed by induction on j. The initial case is j = M, which is 
trivial because vx is a highest weight vector. Next write 

Xsi^ ■ ■ ■ Xs^X_y^ ■ ■ ■ X_y^_^X^X_y^^^ ■ ■ ■ X_^j^jVx = 

(4.6) Xs^ ■ ■ ■ Xs^X^y^ ■ ■ ■ X_^j._iX_^^,_^jX^ ■ ■ ■ X^^j^jVx 

+ Xsj^ ■ ■ ■ Xs^X_^^ ■ ■ ■ X^y._^ [X^, X_^^,_|, J ■ • ■ X^^j^jVx- 



The first summand on the right hand side of ( [4.6| ) is zero by the induction hypothesis on j. 
The second summand is also zero for the following reasons. If C = Ij+i ^^e Claim |^. If 
not, then either ( — 7^+1 is not a root and then [X(^,X_^^,^J = 0, or else ( — 7^+1 is a root. 
In the latter case, if it is a positive root we can use the induction hypothesis on j, and if it 
is a negative root then we can use the induction hypothesis on X. This completes the proof 
of Claim ^ We finally turn to the proof of Claim ||. We can write 

-^(5jv ■ ■ ■ -^(5i-^-7i ■ ■ ■ X^JJ^JVx = 
M 

(4.7) ^ Xsj^ ■ ■ ■ X^jX.^j ■ ■ ■ X_^^._j [X^^ , X_^JX_^^^_^^ ■ ■ ■ X_^^^f A 

i=i 

+ Xsj^ ■ ■ ■ Xs^X^j-^ ■ ■ ■ X^yj^jXs-^vx- 



The second summand in the right hand side of (|4.?1 ) is zero because vx is a highest weight 
vector. The first summand is also zero because if Si = 7^ we can apply Claim |^. Otherwise, 
either 61 — 7^ is not a root and then [X^^, X_^j.] = or else 61 — 7j is a root and then it is a 
positive root by Claim ^ so that we can apply Claim |^. This completes the proof of Claim ^ 
and the proof of the proposition. □ 
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4.8 Lemma Let nx he the complex irreducible representation of G with highest weight A and 
representation space V\. Let fi be a weight of ttx and fix a weight vector v^. Let p denote 
a real form of nx in case ttx is of real type, or the realification of ttx in case it is either of 
quaternionic or of complex type. Suppose that p is of class O^. 

(a) If Tlx is of real type, then 

U\Q^)v,+U\Q^)e{v,) = Vx, 

where e is the real structure on Vx defined by p. In particular, by taking p = X, we find 
that 

U\q'^)vx+U\q'Hvx)=Vx. (C2) 

(b) If Tlx is of complex or quaternionic type, then = Vx- 

Proof (a) Consider the real vector p = + ev^. The second osculating space OpiGp) is 
spanned over R by Xp, XFp, where X, F G g. Taking linear combinations with complex 
coefficients of these vectors, we can write that the complexification 

OliGpf C U\Q'')v,+U\Q'')e{v^) C Vx. 

But p of class (O^) implies that Ol{GpY = Vx- 

(b) Let p = v^. We have that Op{Gp) C U'^{g'^)v^ C Vx as real vector spaces and the 
proof is similar as in (a). □ 

We now state the main result of this section. Notice that the same result is claimed 

in 



10 1, p. 271, with an attempt of a proof which in our opinion is not satisfactory. 



4.9 Proposition Let p be a real (orthogonal) irreducible representation of a compact con- 
nected semisimple Lie group G with complexified Lie algebra q'^, and let nx be the associated 
complex irreducible representation. Suppose that p is of class O^. 

(a) If p is of quaternionic type, then k{X) = 1. 

(b) If p is of complex type, then k{X) = 1, 2. 

(c) If p is of real type, then k{X) = 2, 4. 

Proof. First consider p to be of quaternionic or complex type, i. e. = ttaQtt^. Then p of 
class forces W{q'^)vx = Vx (Lemma ^) and then sqVx G W{q'^)vx implies that A;(A) < 2 
(Proposition [4.4|) . From this follow (a) and (b). 

Now take p to be of real type. Let vx be a highest weight vector of p^ = nx- Since vr^ is 
self- dual, 

lowest weight of tta = —highest weight of vr^ = —A = sgA. 

Also, there exists a C-conjugate linear, G-invariant involution e of Va and t[vx) is a lowest 
weight vector. We shall assume kiX) > 6 and derive a contradiction. Since k{X) is even, it is 
possible to write 

k{X) 

—— = rii + . . . + + m 
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for some integers 0<io<s,0<m< riio+i, where is as in Proposition Set 

/i = A - niPi - ... - riiopig - mpig+i. 
Note that /i is a weight: a /i- weight vector is 



Use Lemma to decompose m = -Ui + ^2 where Ui G a and U2 G W^(g'^)e(f a). First, 

assume that U2 ^ 0. Since > 3, we have 

(4.10) U2 = u-uie U'^{g')vx+U\g'')vx C u'^{g'')vx. 

Moreover, it is clear that U2 may be assumed to be a /i-weight vector (because a component 
of M2 in a different weight space has to cancel with the corresponding component of Ui in the 
same weight space) and that 

for some complex constants c^^s (we do not need to consider terms of first order in the sum 
because X — fi cannot be a root). Therefore, 

7^ (m2,M2) = (m2, 5Z c^,5X^yX_se{vx)) = ( ^ 0^,5X5X^^2, e(t;A))- 

7,(5eA+ 7,5gA+ 



This shows that 



0-^,5X5x^^2 

7,(5eA+ 



is a nonzero multiple of the lowest weight vector e(fA)- This, combined with (^4.10|) , gives 
that e{vx) is in U^~~^'^{g'^)vx- But + 2 < /i;(A), contradicting Proposition 
In case M2 = we have that m = -ui G U'^{g'^)vx- Since 



-PiQ + l -Pio + 2 -Ps ^ ^ 



is a nonzero multiple of the lowest weight vector e(f a) and rijg+i — m + ^4,3+2 + . . . + ns = -^y^, 
again e(fA) G W~2^^^(g'^)f a, contradicting the same proposition. □ 



5 The candidates to a position in class 

In this section we want to elaborate a list of possibilities for real irreducible representations 
of class which we shall use later in Sections ^ and |^ to classify variationally complete and 
taut representations. Our point of view, as is usually the case in other papers on the subject, 
is to classify orthogonal representations up to the following equivalence relation: we call two 
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representations p : G 0(y), p' : G' ^ 0(^0 ifnage equivalent if there exists an isometry 
iIj -.V such that 0(V')(p(G')) = p'iG'), where O(V^) : 0{V) 0{V') is the induced 

conjugation mapQ. In particular, the image equivalence class of p always contains its dual p* 
(in fact any representation obtained from p by an automorphism of the Dynkin diagram), as 
well as the pull back representation p = p o p of a covering group p : G ^ G. 

Let G be a compact connected Lie group. We have that G is finitely covered by T" x 
Gg, where is an n-dimensional torus and Gg is a compact connected semisimple Lie 
group (which may also be assumed to be simply-connected, whenever convenient), and any 
representation of G pulls back to a representation of T" x Gg. In view of image equivalence, 
in order to study representations of G, we may assume that G = T"" x Gg and restrict to 
almost faithful representations. Next we observe that any complex irreducible representation 
of T" is a character (hence of complex type). Therefore a complex irreducible representation 
of G can be of real type or of quaternionic type only if it is trivial on T", and in case it is of 
complex type, then it has an (n — l)-dimensional kernel on T". Now we have come to our 
working hypothesis (for a circle group S^, let denote the nth power representation of that 
circle which is a complex representation of complex type): 

Let p be a real irreducible representation of a compact connected Lie group G on 
a finite-dimensional real vector space V. 

• If p is of real type, we assume that G = Ggis a. compact connected semisimple 
Lie group, and p is a real form of a complex irreducible representation n = ttx 
of G of real type on the complex vector space Vx for some highest weight A. 

• If p is of quaternionic type, we assume that G = Gg, where Gg is as above, 
and p is the realification of a complex irreducible representation vr = tta of G 
of quaternionic type on the complex vector space Vx for some highest weight 
A. 

• If p is of complex type, we assume that G = Gg or G = x Gg, where Gg 
is as above, and p is the realification of a complex irreducible representation 
vr of G of complex type. According to the form of G, we write vr = tta or 
vr = X ® tta as a representation on the complex vector space Va = C (8>c Vx 
for some highest weight A. 

Isotropy representations of symmetric spaces and cohomogeneity one representations of 
compact connected Lie groups are of course two classes of representations that belong to 
class O^. In order to keep the exposition as simple as possible, we assume knowledge of the 
classification of symmetric spaces. So in the course of this section we will be disregarding all 
the isotropy representations of symmetric spaces that we encounter, according to tables 8.11.2 
and 8.11.5 in Moreover, we classify in Section |^ the cohomogeneity one representations 



of compact connected Lie groups. (As a side remark, note that our proof of Theorem |8 
is completely independent from the classification of the symmetric spaces.) Therefore, in 
addition to our previous hypotheses we assume throughout this section: 



•^This is finer than the notion of orbit equivalence which is explained in Section but of course not as 
fine as the usual notion of equivalence for representations. 
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The real irreducible representation p is of class (9^, cohomogeneity greater than 
one and it is not the isotropy representation of a symmetric space. 



The results of our investigation in this section (namely, Propositions |5.1| , |5.2| , |5.17| , |5.18| , 
|5.19| , p.20| and |5.21| ) finally imply: the possibilities for p are those collected in the table of 
Proposition |6.3| and in Tables C.l, C.2, C.3 and C.4 in Appendix C. 



We need some amount of notation (cf. ||4^, p. 237). Let A„ = SU(r2+l), B„ = 
Spin(2r2 + 1), C„ = Sp(n), D„ = Spin(2n), G2, F4, Eg, E7 and Eg refer to E. Cartan's 
classification types of simple Lie groups and Lie algebras. For the complex semisimple Lie 
algebra g'^, we use the following notation for its complex irreducible representation vr^: if the 
integer 2(A, aj) 7^ 0, then we write it next to the vertex of the Dynkin diagram of 

which corresponds to (recall that S = {ai, . . . is the simple root system). For 
example, 



denote respectively the vector representations of A„, B„, C^, D„, and the adjoint represen- 
tations are given by: 



Ai 

A„, n > 2 



Bn, n > 3 



Cn, n>2 



D„, n > 4 



2 

o 



G2 
F4 

Eg 
E7 

Eq 



1 

3j 




Note that we are using the dot convention: if there are two lengths of roots, then the short 
roots are black in the Dynkin diagram. And a real irreducible representation is denoted by 
the diagram of its complexification. 

We add that we shall refer repeatedly to Table B.l in Appendix B, which contains the 
values of the invariant A;(A) for the fundamental representations of the complex simple Lie 
algebras. 



5.1 The case where p is of quaternionic type 

Here there is nothing: 

5.1 Proposition The representation p cannot be of quaternionic type. 
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Proof. If p is of class and quaternionic type, Proposition O says that k{X) = 1. A 
glance at Table B.l now shows that G = Sp(n) and vr^ is the vector representation. But 
then p is of cohomogeneity one. □ 



5.2 The case where p is of complex type 

Here the result is: 

5.2 Proposition Let p be of complex type. Then p is one of the following: 



G 



P 



Gonditions 



SU(n) X SU(m) 
SU(n) X Sp(m) 
51 X SU(n) X Sp(m) 

Spin(lO) 

SO(2) X Spin(7) 
SO(2) X Spin(9) 
SO(2) X G2 



(o o ® o o) 

1 1 
(O O (g) 



(o o © o o) 

1 



— o 



ix ® o — 



-o©x 



-1 



' o — 



-o) 




[x © X'^) © o — o 

1 

{x © X'^) © 



n ^ m 



n> 3, m > 2 



m > 2 



n odd 



The proof of Proposition 5.2 will be given after the proof of the following two lemmas. 



5.3 Lemma Let tt\ he a complex irreducible representation of a compact connected semisim- 
ple Lie group Gg and suppose that the realification of the tensor product representation 
71 = X ®) Tlx of G = X Gs is of class . We have: 

(a) if Tlx is of real type andGg is simple, then Gg = Spin(7) orGs = Spin(9) and 71 x is the 
respective spin representation, or Gs = G2 and tix is the 7-dimensional representation, 
or Gs = SO(m) for m 2, A and ttx is the vector representation; 

(h) if Tlx is of real type and Gs is not simple, then Gs = Sp(l) x Sp(m) and ttx is the 
tensor product of the vector representations of each of the factors; 
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(c) ifiTx is of quaternionic type or complex type, then k{X) = 1, 2. 



Proof, (a) and (b) Let Vx be the representation space of ttx and V be the real subspace 
where a real form acts. Note that the representation space for vr is still Vx, but vr is of complex 
type. Let p = 1 (S> a G C ®r V = = Vx, n the dimension of V and q the codimension of 
GgQ in V. Since the realification of vr is of class O^, we have that Op{Gp) = V^^ = V" © iV. 
But 

OliGp) C 1 ® [Ra + OliCa)] +i^[Ra + T,(G,a)], 

so by comparing real and imaginary parts we get that V = Ra + Ta{Gsa). Therefore q = 1 
and G acts with cohomogeneity one on V. Now (a) follows from Lemmas and ^.14 
below, and (b) follows from Lemmas 5.11 and p.l5| below. 

(c) Consider the point p = wa G Vx- We have that Op{Gp) C U'^{g^)vx C Vx as real 
vector spaces. Since the realification of tt is of class O^, we have that U^{q'^)vx = Vx- 
But U'^{Qg)vx = U'^{g'^)vx, since the circle subgroup preserves the weight spaces of tta. 



Therefore U {g'g)vx = Vx and we can apply Proposition [4.4| as in the first paragraph of the 



proof of Proposition 4.9 



□ 



Part (a) of the following lemma also follows from |^ 



5.4 Lemma (a) Let ttx be one of the following complex irreducible representations of com- 
plex type of the compact connected semisimple Lie group Gg: 



Gs 


TTA 


Gonditions 


SV{n) X SU(m) 
SU(n) 

Spin(lO) 


1 1 

O O (g) o o 

1 


n ^ m 
n = 2p + 1 



Consider the tensor product representation tt = x^ttx of S^xGs- Then the realifications 
of TTx and vr are orbit equivalent. 

(b) The realifications of the following complex irreducible representations tta of complex 
type of the compact connected semisimple Lie group Gg are not of class : 



Gs 


TTA 


Gonditions 


SU(n) 
SU(n) 
SU(n) X SU(n) 


2 

1 

1 1 

O O (g) o o 

1 


n>3 
n = 2p > Q 
n>3 
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Proof. Preliminary considerations: Each one of these representations vr^ has the 
property that the reahfication of vr = x ® vr^ is the isotropy representation of a compact 
irreducible Hermitian symmetric space X = Lq/Kq, where Kq is locally isomorphic to S^xGs- 
We refer to Chapter VIII, §7, for results about Hermitian symmetric spaces that we will 
be using in the following. Let [q = 60 + Po be the decomposition of the Lie algebra [q of Lq 
into the ±l-eigenspaces of the symmetry. Let Cq be the Lie algebra of 5*^, and let to be some 
Cartan subalgebra of the Lie algebra of Gg- Then i)^ = Cq + to is a Cartan subalgebra of 
60 and of to- Let I be the complexification of Iq and let c, t, f), 6, p be the complex subspaces 
of [ spanned by Co, to, f)o, ^O; Po- Let A denote the root system of ([, ()) and consider the root 
space decomposition I = i) + J2aeA ^a- For each a G A, we have that either [q, C 6 or [q, C p, 
in which cases the root a is called respectively compact or noncompact. A root is compact if 
and only if it vanishes on c. Let Ac and A„ denote respectively the subsets of A of compact 
and noncompact roots. Then we have decompositions t = () + J2a(^A^ ^a, P = S^eA„ ^- Each 
root is real- valued on ii)^. We introduce a lexicographic ordering on the dual of ii)^ that takes 
ico before ito- Let A"*", A+ and A+ be the set of positive roots in A, Ac and A„, and define 

p+ = 5^ i„ p-= Yl ^• 

7eAi -76A+ 

Then p = p+ + p~ is an ad^-invariant decomposition. The ad^^-invariant complex structure 
Jo on Po is given by adj^g, where Hq G Co is determined by 7(-ffo) = i for 7 G A^. The 
C-linear extension J of Jo to p has p^ as ±2-eigenspaces. Let F = {71, . . . ,7s} C A+ be a 
maximal subset of strongly orthogonal roots. Then ao = Y2^j=i ^(^7^ ~ -^-ij ) ^ maximal 
Abelian subspace of po. We view as the adjoint action of Kq on pQ, and then vr'" is polar 
and Qo is a section for vr^. We also view vr as the the adjoint action of Kq on p"*", and then 
the weight system of tt is A^. Now the root system of {q^, t) is 

Ao = {a\i : a G Ac} (also = {a\i : a G A+}), 

and the weight system of tta is 

$ = {7lt:7e A+}. 

It is also useful to remark that any adg^^ -invariant inner product < ■ , ■ > on po is given by 
a negative multiple of the restriction of the Killing form of lo; fix one and extend it to a 
Hermitian product ^ ■ , ■ ^ on p. Then « ■ , ■ ^ is ad^^-invariant and J is a skew-Hermitian 
operator on p with respect to ^ ■ , • 

Proof of (a): Since ao is a section for vr^, it is clearly enough to check that given 

s 

where aj G R, there exists H G ito such that 

(5.5) iadnP = JoP- 
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Write H = J2aeA+ (^aHa, where Ha are the coroots. Then (|5.5| ) is equivalent to the following 
system of linear equations: 

aGA+ 

which we can solve in each one of the three cases with the help of the tables in Appendix A. 

1 1 

Let Gs = SlJ{n) x SU(m), n < m, nx : O O (g) O O. Then Lq is SU(?7. + m), 

Aq = {6i — 6j : 1 < i < j < n ot n + 1 < i < j < n + m}, 

^ = {6i — 6j : 1 < i < n and n + l<j<n + m} 
and r = {0,- On : 1 < i < ri}. Here we can take i7 to be a suitable multiple of 

1 

Let Gs = SU(n), n odd, tta : O — O O. Then Lq is SO(2n), = {9, - 9^ : 1 < i < 

j < n}, ^ = {9i + 9j : 1 < i < j < n} and F = {6'2i-i + 92i : I < i < Here we can take 

to be a suitable multiple of Hg-^^e^ + Hg^^e^ + . . . + He^_^_e^. 

Let Gs = Spin(lO), tta : o — o — . Then Lq is Eq, A+ = {±9,+9j : 1 < i < j < 5}, 

$ = iU^s -9r-9, + eA) : Ht^e, = +1} and F = {i(^8 - e, - 9, + 9, + 9, + 9, + 92 + 
|(^8 — 6*7 — 6*6 + 6*5 — 6*4 — 6*3 — 6*2 — ^^i)}- Here we can take if to be a suitable multiple 
of Hg^+0^ — Hg^^e^. This completes the proof of (a). 

Proof of (b): We will verify directly that the second osculating space at the point 

s 

is not of maximal dimension. More precisely, we shall see that 

(5.6) <adxP,^oP> = <adyadxP , ^oP> = 0, 

for all X, y G g,,. Notice that JqP = Yli]=i ^(-^7j + ^-ij) ^ Po- 

It suffices to prove the identities ( |5.6D for the basis elements ( [4. 3D of Qs- We have that 
<adxP,^oP> = 3fJ«adxP,^P^ and < adyad^p , Jop> = — 3?«adxp, adyJp^. The 
only cases where the real parts of the Hermitian products are not obviously zero are the 
following (a, (3 E A^): 

s 

« adiH^p ,Jp::>> = "Y ^ ad/^^ (X^^ - X_^^ ) , + X_^^ » 

j,k=i 

s 

-Ik) ^ 

j,k=l 

s s 

(5.7) = 2 5^7,(^^^ = 2(5^7,, a), 

i=i i=i 
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assuming the root vectors to have length one. We will show below that the last expression 



in (|5.7|) vanishes for the representations we are interested. 

«adi(x,+x„„)P,^P» = «adx^+x-^P,T.l=iiXik+^~jk)» 

s 

j,k=l 

(5.8) = 0, 

because adx„-^7j- is either a (7^- + a;)-root vector or 0; ±7^ ± 7^ ^ A; and root spaces 
corresponding to distinct roots are orthogonal. 



s 



« adiH^p , adx^-x_^ Jp) » = ^ (^^q) « (^7, + X_^. ) , adx^_x_^ (^7, + X_^^ ) » 

j,k=i 
= 0, 

for the same reason why (|5.8|) is zero. 

3?«adx„-x_„-^P,adj(x^+x_^)P» = ^<^adx^Jp ,iadx^p» 

—^<K adx_„ Jp , iadx_i^p ^ 
+3?« adxa Jp ) i^dx_i^p y> 
— adx_„ Jp , iadxfjP y> 
= 3ft« Jp,iad[x_^,Xc]P» 
(5.9) Jp,iad[x^,xjP», 



because, for instance, 



«adx,Jp,«adx^P» = « adx_^adx, , 

= « Jadx_^ adx^p , ip » 

= -«adx_^adx,P, 

= -« adx_.^adx„P , iJp » 

= « iadx_^adx^p , » 

= <^Jp, ia.dx_g adxc,P » • 



Now ( |5.9| ) shows that everything amounts to checking that ( |5.7| ) vanishes. Next we do 
that in each one of the four cases. Let 7 = Yl'j=i Ij- Then the vanishing of ( |5.7|) is equivalent 
to 

(5.10) (7,A+)=0. 

2 

Let Gs = SU(n), tta : O O. Then Lq is Sp(n), = {9i - 9j : 1 < i < j < n}, 

$ = {Oi+Oj ■.l<i<j< n}U{2e^ : 1 < ^ < n}, r = {26^ : 1 < i < n} and 7 = 2^i + . . .+2^„. 
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Let Gs = SU(n), n even, tta : 



Then Lq is SO(2n), A+ = {6,-6^ : 1 < i < 



J <n},^ = {e^ + eJ : 1 < i < J < n}, r = {^2^-l + ^2^ : 1 < 2 < f } and 7 = 61 + 62 + .. . + 6^. 

1 1 

Let Gs = SU(n) x SU{n), tta : O O ® O O. Then Lq is SU(2n), A^^ = {^^ - 

6j : 1 < i < j < n or n + 1 < i < j < 2n}, $ = {0,; - 0j : 1 < i < n and n + 1 < j < 2n}, 
r = {6i - 62n+i-i ■.l<i<n} and = 61 + 62 + ■ ■ ■ + 6n - 6^+1 - 6^+2 - ... - 6'2„. 



Let = Eg, : 




. Then Ln 



E7, 



A+ = {±6, + 6, 



l<^<J<5}U{-{6s-6r-6e + J2 ^^^0 



+1}, 



1 ^ 

$ = {^^8 - ^7} U {±6, + 6,:l<i<5}U{-{6s-6r + 6, + J2 ■ ^li^i = -1), 



i=l 



T = {6s- 6-r, 6, + 6,, 6^ - 6,} and ^ = 6s - 6j + 26,. 

It is immediate to see that ( |5.1CI| ) holds in each case, and this completes the proof of (b). 

□ 



We know from Proposition ^]9| and Lemma ^]3| that k{X) 



Proof of Proposition |57 
2. In particular, Gg can have at most two simple factors. 

Suppose A;(A) = 1. This implies Gg is simple. In the case G = Gs, we have vr is of complex 
type, so G = SU(n) and vr is the vector representation. In the case G = x Gg, we have tta 
is of quaternionic or of complex type, so Gg = Sp(n) or Gs = SU(n) and tta is the respective 
vector representation. This only gives representations with cohomogeneity one. 

Next suppose /c(A) = 2. The first case is Gs = Gi x G2, where Gi is a simple group. Here 
tta decomposes as an outer tensor product tt^^ (g) tt^^ ? where vr^. is the representation of Gi of 
highest weight fii with k{fii) = 1. This forces each Gi to be either Sp(n) or SU(r;,) for some 
n, and vr^- to be the corresponding vector representation. Using Lemma |5.3| , item (b), and 



Lemma [5.4| , item (b), we get the first three representations in the table. 

The second case is Gs is a simple group. For G = Gs, an inspection of Table B.l reveals 
four possibilities for tta, namely: 




and 




Now Lemma 5^, item (b), restricts these to the next two representations in the table. 

For G = X Gs, if 7i\ is of complex type, we only get representations associated to 
symmetric spaces, and if tta is of real type. Lemma item (a), gives the last three repre- 
sentations in our table. □ 



5.3 The case where p is of real type 



The real case is much more involved than the previous two. We know from Proposition ^ 
that k{X) = 2, 4. In particular, G can have at most four simple factors. 
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As a first step towards the classification in the case p is of real type, we introduce the fol- 
lowing variations of condition (C2) of Lemma [4 .81 for a representation Tr;^ with representation 
space V\ and highest weight vector v\: 





= Vx 






= Vx 




U\Q-)v^+U\Q')e{v^) 


= Vx 


(Ci) 


Wi(s>A + CeK) 


= Vx 






= Vx 





It follows from the identity eXa = X_a^ that conditions {C^i) and (C'l) are equivalent, and 

that conditions (Ci) and (Ci) are equivalent. 

2 2 

5.11 Lemma The following conditions are equivalent for a complex irreducible representa- 
tion TTx of real type of a compact connected semisimple Lie group G, with invariant real form 
V: 

(a) condition (Ci); 

(b) G is transitive on the unit sphere ofV; 

(c) G acts with cohomogeneity 1 on V; 

(d) {Xp : X G 0} = (Rp)^ for all p G V ; 

(e) {Xp : X G 0} = (Rp)^ for some p eV. 

Proof (b), (c), (d), and (e) are clearly equivalent. We prove that (a) is equivalent to (e). 
In fact, let p = vx + e{vx)- Clearly {Xp : X G g} C (Rp)^. Using the basis ([4.3|) , we find 
that {Xp : X G qY is spanned over C by 

vx-e{vx), X^aVx, Xae{vx), a G A+. 

But condition (Ci) holds precisely when this is also a system of generators for (Rp)-^'^ over C. 

□ 

5.12 Lemma The following conditions are equivalent for a complex representation ttx of 
quaternionic or complex type of a compact connected semisimple Lie group G on Vx, with 
highest weight Vx: 

(a) U\g'^)vx = Vx; 

(b) G is transitive on the unit sphere ofV{; 

(c) G acts with cohomogeneity 1 on V^; 

(d) {Xp : X G 0} = (Rp)^ for all p G V{; 
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(e) {Xp : X G 0} = (Rp) for some p e VJ^ . 
Proof. The proof is analogous to the proof of Lemma p-ll\ 



□ 



5.13 Lemma The self-dual complex irreducible representations of compact connected simple 
Lie groups satisfying condition (Ci) are precisely^: the vector representation of SO (m) for 
m ^ 2, 4, the vector representation o/Sp(m), the 7-dimensional representation 0/G2, and 
the spin representation o/Spin(7). 

We will prove Lemma |5.13| together with the next lemma. 

5.14 Lemma The self-dual complex irreducible representations of compact connected simple 
Lie groups satisfying condition (Ci) are, besides those listed in Lemma |5.13| ; precisely the 
following: 



G 




kiX) 


SU(2) 


3 




3 


SU(6) 


1 


3 


Spin(9) 


1 


2 


Spin(ll) 


1 


3 


Spin(12) 




3 


Sp(3) 


1 

• — m=o 

1 


3 


E7 
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Proof of Lemmas ^.iSj and \5. 1^ . It is obvious that condition (Ci) implies condition (Ci). 
So we consider (Ci). It implies the following: 

(a) k{\) < 3. 

(b) The dimension of tta is bounded by dimg*^ — rankg'^ + 2. 

(c) If is a weight, then its multiplicity is one. In particular, if tcx is the adjoint represen- 
tation, then the rank is at most one. 



In fact, (a) follows from the fact that k(X) > 4 contradicts Proposition [4.4| . For (b) notice 
that the dimension of the left hand side of (Ci) is bounded by 2(number of positive roots) + 2. 
Finally, if is a weight, then the 0- weight space is contained in U^{q'^)vx (because eXa = 
X_„e), and from this we see that A is a root and the 0-weight space is in fact spanned by 
X_xV\, which implies (c). 

"'The half-spin representations of Spin(8) are image equivalent to the vector representation. 
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We will use the above observations to complete the proof of the lemmas. 
Let TTx be of real type. Then k{X) = 2. The adjoint representations of A„, B„, C„, D„ 
with n > 2, and of G2, F4, Eg, E7, Eg are immediately excluded. We run through all the 

remaining cases of representations of real type with k{X) = 2. 

2 1 

Let G be of A„-type. Then tcx is o or o — o — o which are cited in the lemmas. 

1 11 

Let G be of B„-type, n > 2. Then ttx is o — •■■ — o • or o — o • or o — o — o 

All of them are cited in the lemmas. ^ 

Let G be of C„-type, n > 3. Then ttx is • — • — ■■• — • o, which cannot occur because 
the multiplicity of the 0- weight is n — 1. 

Let G be of D„-type, n > 4. Then tcx is o — ■•• — cf , which is cited in the lemmas. 

1 1 
Let G be of exceptional type. Then tta is • o or o — o • — •. Only the former is 

cited in the lemmas; the latter has as a weight of multiplicity 2. 

Now let Tlx be of quaternionic type. Then fc(A) = 1 or k{X) = 3. The case k{X) = 1 gives 

the vector representation of Sp(m), which is cited in the lemmas. In the case k{X) = 3, the 

only possibilities not cited in the lemmas are the following four representations, which we 

next eliminate: 

1 

• G = Be, TTA : o — o — o — o — (>=•■. here A = K^i + 62 + 63 + 64 + 65 + Oq), fi = 
^2 + ^3 ^ ^4 ~ ^5 — ^e) is a weight and A ± /i are not roots which violates 



'1 



condition (C 



In the remaining cases we make use of the bound on the dimension of the representation (see 
e. g. Table 5 in the Reference Chapter of ||33[|): 



3 



G = C„, tta : • — • •=^: dimvTA = |n(?2+l)(2n+l) > 2^2+2 = dimC„-n+2, 

for n > 2; 

1 1 

G = C„, tta : • • •^D: dimvTA = + > 2n'^ + 2 = dimC„-ra + 2, 

for n > 2] 

1 

G = C„, TTA : • • • •=^: dimTTA = - 2){2n + 1) > 277,^ + 2 = 

dim Cn — n + 2, for n > 4: 



On the other hand, the representations listed in the statements of the Lemmas |5.13| and p. 14 



do satisfy (Ci), as can be easily checked using the fact that each of them has all weights with 
multiplicities one. This concludes the classification of self-dual representations that satisfy 

(Ci). As for condition (Ci), we observe that (Ci) implies that k{\) < 2. This eliminates all 

2 2 ^ 

representations in the table of Lemma |5.14| except o — o — o that one can be eliminated 
by a direct check. Finally it is easy to verify that the representations cited in Lemma p. 13 



do satisfy (Ci). □ 
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5.15 Lemma The only complex irreducible representation of real type of a compact connected 
semisimple nonsimple Lie group G satisfying condition (Ci) is the tensor product of the vector 
representations o/Sp(l) andSp{m). 



Proof. We already know that A;(A) = 2 (cf. proof of Lemmas ^.131 and |5.14|) , so G = 
Gi X G2, where Gi is a simple group, and tta = tt^^ ®7r^2 with k{fii) = 1. Since each 7r„- is self- 
dual, it must be of quaternionic type. Therefore, each Gi = Sp{ni) and the corresponding vr^. 
is just the vector representation. Now one of the rij must be 1, for otherwise condition (Ci) for 
TTx is violated by consideration of a weight vector of type f 1 f2, where Vi is an intermediate 



(that is, neither highest nor lowest) weight vector for vr^ 



Mi- 



□ 



5.16 Lemma The complex irreducible representations ttx of quaternionic type of compact 
connected simple Lie groups G satisfying k{X) = 3 and condition {Cii), but not condi- 
tion (Ci), are the following: 



G 




Spin(13) 
Sp(2) 


1 

1 1 



We postpone the proof of Lemma 5.16 to the end of Subsection 5.3.5 , since the methods 
used to prove it better belong there. 



5.3.1 The case where G has four simple factors 

In this case we have: 

5.17 Proposition Let p be of real type and G = Gi x G2 x G3 x G4, where Gi is a simple 
group. Then G = SU(2) x SU(2) x SU(2) x Sp(ri), n > 2, and vr is the tensor product of 
the vector representations of each of the factors. 

Proof Set TTx = 7r^i®7r^J2®7r^3®7r^4) where vr^. is the representation of Gi of highest weight 
/ij. We have k{pi) = 1 for all i, and since each vr^^ is self-dual, it must be of quaternionic 
type. Therefore, each Gi = Sp(nj) and the corresponding vr^. is just the vector representation. 
Notice that = 1 for all i gives the isotropy representation of a symmetric space. Suppose 
now that for two indices, say 3 and 4, we have > 1 and n4 > 1. Take weight vectors Vi of 
TT^^ such that: Vi is the highest weight vector of vr^^, V2 is the lowest weight vector of vr^j? 
f 3 and f 4 are intermediate weight vectors of vr^g and Hf^^ , respectively. Then f 1 f 2 ® f 3 (S> f 4 
is a weight vector of tta which cannot be contained in U'^{q'^)vx +W^{Q'^)e{vx), violating the 
condition (C2) from Lemma |4.8|. □ 



5.3.2 The case where G has three simple factors 

The classification is: 
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5.18 Proposition Let p be of real type and G 
Then p is one of the following: 



Gi X 6*2 X G^, where Gi is a simple group. 



G 


P 


Conditions 


SU(2) X Sp(n) X SO(2m) 




11 1 


m > 3, n > 2 


SU(2) X Sp{n) X SO(2m+ 1) 


11 1 


n>2 


SU(2) X Sp(n) X G2 


11 1 




SU(2) X Sp{n) X Spin(7) 


11 1 

(g) • — •■■ — (g) — c^=» 




SU(2) X SU(2) X Spin(9) 


11 1 





Proof Let tix = tt^^ (g) n^^ (g) n^^ , where n^. is the representation of Gi of highest weight 
Pi, say with k{pi) = k{p2) = 1, kl^ps) = 2. Then tt^^, tt^j each of quaternionic type 
and 7r^3 is of real type. So each of vr^^, vr^j is the vector representation of Sp(m) for some 
m. Moreover, one of them must have m = 1, say vr^^, for otherwise condition (C2) for ttx is 
violated by consideration of a weight vector of type f 1 f 2 ® fs, where Vi is an intermediate 
weight vector for vr^^. Now we have Gi = SU(2), G2 = Sp{n), and tt^^, n^^ the respective 
vector representations. Next take a weight vector f 1 f 2 fs for vr^ such that Vi is highest, 
V2 is lowest and is intermediate. Condition (C2) for tta forces vr^g to satisfy condition (Ci). 
If n > 2, we could also take V2 to be intermediate, and then condition (C2) for ttx implies 
something stronger, namely that vr^g must satisfy condition (Ci). Our table now follows 
from Lemmas |5.13| and □ 



5.3.3 The case where G has two simple factors and p has two factors of real 
type 

Here the result is: 

5.19 Proposition Let p be of real type and G = GiX G2, where Gi is a simple group. Write 
p'^ = TTx, decompose vta = vr^i ® vr^j suppose that each vr^,. is of real type. Then each iifj,. 
is one of the following representations : the vector representation o/SO(m) for m ^ 2, 4, or 
the 7-dimensional representation of G2, or the spin representation of Spin(7), or the spin 
representation o/Spin(9); but the case where Gi = G2 = Spin(9) is excluded. 

Proof Here k{pi) = k{p2) = 2. Let vi (g f 2 be a weight vector for ttx such that each Vi is 
an intermediate weight vector for tt^-. Condition (C2) for ttx applied to vi (8 V2 forces each 
TT^,. to satisfy (Ci). In fact, one of the vr^- must even satisfy (Ci). The claim follows from 
Lemmas |5.13| and [5.14| . □ 



5.3.4 The case where G has two simple factors and p has two factors of quater- 
nionic type 



The classification is: 
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5.20 Proposition Let p be of real type and G = GiX G2, where Gi is a simple group. Write 
= TTx, decompose ttx = tt^^ ® vr^^ '^'^'^ suppose that each vr^. is of quaternionic type. Then 
p is one of the following: 



G 


P 


Gonditions 


Sp(n) X SU(2) 


1 3 


n>2 


Sp(n) X SU(6) 


1 1 


n>2 


Sp(n) X Spin(ll) 


1 1 




Sp(n) X Spin(12) 




n>2 


Sp(l) X Spin(13) 


1 1 




Sp(l) X Sp(2) 


111 




Sp(n) X Sp(3) 


1 1 

1 


n>2 


Sp(n) X E7 




n>2 



Proof. If A;(A) = 2, then k{pi) = k{p2) = 1 and each vr^^ must be the vector representation 
of Sp(nj) for some Ui, so we get a representation associated to a symmetric space. In the case 
k{X) = 4 we can write k{pi) = 1 and k{p2) = 3. Now tt^^ must be the vector representation 
of Sp(?2) and there are two cases to consider: 



n = 1: here tt^j must satisfy condition (C'^ 
n >2: here tt^j must satisfy the stronger condition (Ci). 



The claim now follows from Lemmas 5.14 and 5.16 



□ 



5.3.5 The case where G is a simple group 

The representations of real type of simple groups that are candidates to a position in class 
02 are: 



5.21 Proposition Let p be of real type and suppose G is a simple group. Then p is one of 
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the following: 



G 


P 


Spin(7) 
Spin(9) 
Spin(15) 
Spin(17) 


1 1 
o (>=• 

1 1 

1 

1 



We will accomplish the proof of the above proposition by running through all the cases 
of representations of real type of a simple group with A;(A) < 4 given in Table B.l. In fact, 
according to that table, all representations with k{X) = 2 are either associated to symmetric 
spaces or of cohomogeneity one. So we only need to worry about the case k{\) = 4. A careful 
inspection of Table B.l shows: 

5.22 Lemma Let tta be a representation of real type of a compact connected simple Lie group 
G with k{X) = 4. Then {G, tta) is one of the following: 





G 




Gonditions 


1 


Ai 


4 





2 


A3 


2 




3 


A3 


111 




4 


Ar 


1 




5 


An 


2 2 


n>2 


6 


An 


1 1 


n>A 
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G 



Conditions 



9 

10 
11 
12 
13 
14 
15 
16 
17 
18 
19 



B3 
B3 
B4 
B4 
B4 

By 

Bs 
B„ 
B„ 
B„ 
B„ 

Br, 



o — o 

1 



2 



o — o 



1 



1 




— 



— 



n>2 

n > 3 

n > 3 

n > 4 

n > 5 





G 




Conditions 


20 






4 








n>2 




• 

2 




• 




21 


Cn 












n>2 


1 


• 




• 




1 


22 


Cn 












n>3 


• 




• 




• 


23 


Cn 






2 






n>3 


•- 




• 




• 


24 


c. 










1 


II > 1 


• — 


• 




• 





38 




39 



33 
34 
35 
36 
37 
38 
39 

40 
41 
42 
43 
44 
45 



G 

G2 
G2 
G2 

F4 
F4 
F4 
F4 

Eg 
Eg 
E7 
E7 
Eg 

Ea 



33 
2 

3D 
1 

3D 




To begin with, we note the representations hsted in the tables of Lemma ^.22 associated 
to a symmetric space, namely, numbers 1, 2, 4, 15, 24 (n = 4), 27 and 28. 

5.23 Lemma Let ttx be a representation of real type with k{X) = 4. Assume that X = 
|(3/5j + Pj) where Pi, [3j G B are distinct long roots and that there exists a Weyl group 
involution s such that s(3i = jSj. Then a real form of nx is not of class O^. 

Proof We have that /i = ss/j^(A) = — SPj) is a weight and X — fi = Pi + 2Pj, 
A + yU = 2/5j — Pj. Since Pi and Pj are orthogonal, we get that ||A ± = 5 (normalizing 
the length of a long root to be 1). On the other hand, for a', a" G A we have that 
I la' + a"| p < (I la'l I + | |)^ < 4, so A ± is neither a root nor a sum of two roots. Thus a 
/i-weight vector is not in U^{g'^)vx +U'^{Q^)e{vx), violating condition (C2). □ 

One can use the last lemma together with the tables in Appendix A to get rid of the 



representations listed in Lemma 5.22 under the numbers 3, 12, 17, 21, 30 and 37. 
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We next eliminate the representations ttx with A = 2 (highest root) = 2/?i = |(4/?i), using 
the following observations. 

5.24 Lemma Let tix be a representation of real type with A = 2(3i and assume there is a 
nonzero weight fi such that {X, fJ^) = 0. Then a real form ofiTx is not of class . 

Proof. We check that condition (C2) is violated. In fact, ||A ± = ||A|| + ||yu|P > 
= 4 (normalizing the length of a long root to be 1), but a', a" G A implies that 
I la' + a;"|p < (||a;'|| + ||a;"||)^ < 4, and we are done as in the proof of Lemma [5.23| . □ 

5.25 Corollary Let tt\ be a representation of real type with A = 2j3i. If there is a long root 
Pj G B with Pj 7^ Pi, then a real form of nx is not of class . 

Proof. For in this case — 2/3i, — 0, 2/3i is the maximal /3i-string through 2/5i, so Pi 
is a weight, Pj is a weight, too {Pi and Pj are long roots, hence in the same Weyl orbit, see 
e. g. [|, Ch. VI, § 1, no. 1.3, Proposition 11) and {pi.Pj) = 0. □ 



We use Corollary p.25| to discard the representations listed in the tables of Lemma p. 22 



under the numbers 5 (ra > 3), 16, 20, 29, 36, 41, 42 and 45. And we use Lemma ^.24| with 



/i = —ai + 0^2 (resp. ^ = P2 = ai) to eliminate representation number 5 for n = 2 (resp. 
number 34). There still remain 20 representations to be eliminated in the tables, namely 
numbers 6, 7, 9, 10, 18, 19, 22, 23, 24 (n > 5), 25, 26, 31, 32, 33, 35, 38, 39, 40, 43 and 
44, which in the following we go on to analyze case by case, but before that we want to 
reformulate an argument that has already been used and that will frequently be used in the 
sequel. 

5.26 Lemma Let -kx be a representation of real type and suppose that its highest weight A 
can be written as a root or as a sum of two roots in N different ways (where the order of the 
summands is not important). If is a weight and its multiplicity is bigger than N, then a 
real form of ttx is not of class . 

Proof. Write vx for a highest weight vector of vr^. If condition (C2) is satisfied, then the 
0-weight space must be contained in U'^{Q'^)vx+l^^{Q'^)e{vx). The identity eXa = X_ae shows 
that the 0-weight space must in fact be contained in U^{q'^)vx. But the complex dimension 
of the intersection of the 0-weight space with U'^{q'^)vx is at most A^. □ 

In the remainder of this section several decompositions of tensor products of representa- 
tions will be used, all of which can be found in Table 5 in the Reference Chapter of 



5.27 Lemma Representation number 6 is not of class O . 

Proof. We label the simple roots of A„ as o — o — •■■ — o, where ai = 6i — 6i+i. Then 

the root system is A = {±{6i ~ 6j) : 1 < i < j < n + 1} and the highest weight of 

1 1 

o — o — ••■ — o — o is A = A2 + A„_i = 9i + 02 — On — On+i, so it can decomposed as a sum of 
two roots in exactly two ways: 

A = (^^1 — On+l) + {02 — 0n) 
= {01 — 0n) + {02 — 0n+l)- 
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On the other hand, we find that: 



5 (trivial), 

from where we deduce that the multiphcity of the 0-weight in 7i\ is —n—1 = > 2, 

so we can apply Lemma |5.26| . □ 

5.28 Lemma Representations numbers 18, 10 and 19 are not of class . 
Proof. We have that 

2 1 
1 

= A^fo — o — o — o c^»), n> 5. 



This shows that the multiplicity of the 0-weight is, respectively, n, 6 and (2) (because the 

1 

vector representation o — •■■ — o • has weights 9i, 62, ■ ■ ■ , 9n, 0, —9n, —9n-i, • • • , — 6*1)- But 
the integer = 3 in all cases. □ 

5.29 Lemma Representation number 9 is not of class . 

Proof The highest weight is A = \{39i + 3^2 + ^3)- Now fi = §(3^1 - 3^2 + ^3) is a weight 
and A + yU = 39i + ^3, A — /i = 3^^2 are neither a root nor a sum of two roots, so we are done 



as in the proof of Lemma 5.23. □ 



5.30 Lemma Representations numbers 22 and 23 are not of class . Representation num- 
ber 24 is not of class if n > 5. 

Proof We first observe that: 

1 

• the vector representation C„ : • — • — ■■• — • o has weights ±9i with multiplicity 1; 

1 1 

• the representation • — • — ••• — • o is a subrepresentation of A^(» — ■•• — • o ) and 

has weights ±9i ± 9j with multiplicity 1, and with multiplicity n—1; 

1 1 

• the representation • — • — • — ■•• — • o is a subrepresentation of A^(« — ••• — • o ) 

and has weights ±9i ± 9j ± 9j: with multiplicity 1, and ±9i with multiplicity n — 2; 

1 

• the representation • — • — • — • — ■•• — • o is a subrepresentation of 

and has weights ±9i ± 9j ± 9^ ± 9i with multiplicity 1, and ±9i ± 9j with multiplicity 
n — 1, and with multiplicity (2) — n. 
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If n > 5 then the multiphcity of the 0-weight in • — • — • — • — •■■ — • o is ^n{n — 3) > 
5. Since X = 6i + 62 + 63 + Oa can be written as a root or as a sum of two roots in = 3 
ways only, namely, 

= (ei + ^4) + (^2 + ^3), 



Lemma 5.26 says that • — • — • — • — ■•■ — • o , n > 5, cannot be of class O^. 



Next, it follows from 



1 1 
• — • — • — ••• — (g) • — • — • 

11 1 



n> A, 



that the multiplicity of the 0-weight in • — • — • — •■• — • — O, n > 3, is given by ^{n — 
2){3n — 1) > 4. But A = 26i + 62 + O3 can be written as a root or as sum of two roots in 
N = 2 ways only: 

A = (^1 + ^2) + (^1 + ^3) 
= 2^1 + (^2 + ^3). 



Lemma p.26| now implies that • — • — • — ■•• — • o, n > 3, is not of class O . 



1 



One can also see that S^{9 — • — ••• — • o ), n > 3, has as a weight of multiplicity 
^n{n — 1). Now 



2 

1 2 

^2/ 



S\m — •=o) = m — © • — •=<D © (trivial), 

S\m^ = 

2 1 

• — • © • — • •=o © • — • — • — • © (trivial), n > 4, 



1 



so the multiplicity of the 0-weight in • — • — ■■■ — • o, n > 3, is n{n — 1) > 6. But 

A = 26i + 262 can be written as a root or as sum of two roots in = 2 ways only: 

A = (^1 + ^2) + (^1 + ^2) 
= 2^1 + 2^2- 



Lemma |5.26| implies again that • — • — •■■ — • o , n > 3, is not of class O^. □ 
5.31 Lemma Representations numbers 26 and 32 are not of class O^. 
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Proof. These representations are just 

D„:A^((i ), n>5, 

where o — ■•■ — cf is the vector representation. Therefore the muhiphcity of the 0-weight 
is (2) > 10. But the highest weight \ = 9i + 62 + 03 + 0^ decomposes as a sum of two roots 



in = 3 ways, so we can use Lemma |5.26| . □ 

5.32 Lemma Representation number 35 is not of class O^. 

Proof Here A = Ai + A2 = 5ai + 3a2 decomposes only as (2ai + 02) + (3ai + 202), so 
that = 1. Now we find that: 

1 1112 1 



where • o is the vector representation and 5'^( « o ) = • o © (trivial). Therefore the 

1 1 

multiplicity of the 0-weight in • o is 4 and we can use Lemma p.2(j| . □ 
5.33 Lemma Representations numbers 38 and 39 are not of class O^. 

1 

Proof. First we observe that the 2 6- dimensional representation F4 : o — o • — • has 

weights ^{±9i±92±93±9/i) and ±9i with multiphcity 1, and with multiplicity 2. Therefore 

1 

A^(o — o • — •) has as a weight of multiplicity 13. Now 



1 1 
where o — o • — • is the adjoint representation. It follows that vr^ : o — o • — • has 

as a weight of multiplicity 9. But its highest weight A = ^{39i + 92 + 9^ + 9^) is not a root 
and decomposes as a sum of two roots as follows: 

1 
1 

= 2' 

1 

= 2' 



+ 92 


+ 9^ 


+ 9,) 


+ 9i 


-92 


+ 9^ 


+ 9,) 


+ {9i + 92 


+ 92 


-9s 


+ 9,) 


+ {9i + 9s 


+ 92 


+ 93 


-9^) 


+ {9i + 9i 



Thus the integer of Lemma ^.26| is A^ = 4 and o — o • — • is out. 



Similarly, the multiplicity of the weight in 5*^(0 — o • — •) is 15, and 

1 21 
S'^ip — o=m — •) = o — c^=» — • © o — o=9 — • © (trivial). 
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It follows that tta : 



has as a weight of multiplicity 12. But its highest weight 



A = 29i is not a root and decomposes as a sum of two roots as follows: 

% - 62) 



h + 02) 



'1 



Thus the integer of Lemma p.26| is = 4 and 



-• is out, too. 



□ 



5.34 Lemma Representation number 40 is not of class . 
Proof. We have that 



and 



are respectively the 27-dimensional representation and its dual. Therefore the multiplicity 
of the 0-weight in the tensor product of these represent aions is at least 27. But 







) (trivial) 

where the second summand on the right hand side is the adjoint representation. It follows 



that the multiplicity of the 0-weight in tt^ : 



A 




is at least 20. But its highest 



X = 0s- 


-Oj- 


-0i 


; + is not a root and decomposes as 


a sum of two roots as follows: 


liOs- 


O7- 


e. 


+ + 0,i + 9^ + 92 + 9i) 


+ 1{9s-9t- 


- 9q + 9^ — 94 ~ 9^ — 92 — 9i) 




O7- 


0, 


+ 9, + 9, + 9,-92-9^) 


+ ^{9s-9,- 


- ^6 + ^5 — ^4 — ^3 + ^2 + ^1) 


l(Os- 




0, 


+ 9, + 9^-9, + 92-9i) 


+ li9s-9j- 


- ^6 + ^5 — ^'4 + ^^3 — ^2 + ^^l) 




07- 




+ 9,-9, + 9s + 92-9i) 


+ li98-9-r- 


- 9q + 9^ + 94 — 93 — 92 + 9i). 



Thus the integer of Lemma f).2Q\ is = 4 and we are done. □ 
5.35 Lemma Representation number A3 is not of class O^. 



Proof. We have that 



is the 5 6- dimensional representation. It is 

self-dual and does not have as a weight. Therefore the multiplicity of the 0-weight in the 
exterior square of this representation is 28. But 






(trivial). 
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Therefore the multiphcity of the 0-weight in ttx is 27. But its highest weight A = 6*8— ^^7+ 6*6 +6*5 
is not a root and one can easily check that it decomposes as a sum of two roots in exactly 
= 17 different ways. □ 



5.36 Lemma Representation number 44 is not of class . 

Proof. Here we slightly change the argument used in the previous lemmas. The highest 
weight A = 2618 and (A, 6*8 - 6*7) 7^ 0, so z/ = A - {6s - 67) = + 67 is a weight. Now Og + 67 
and 6*6 + are long roots of E8, hence in the same Weyl orbit. It follows that yU = 6*6 + ^5 is 
a weight, too. Observe that each of A ± yU is not a root and can be decomposed as a sum of 
two roots in only one way: 



The proof will be complete if we show that the multiplicity of /i as a weight is greater than 
2, since then we will have that condition (C2) is violated. 

In fact, the multiplicity of /i is the same as the multiplicity of u. We use Freudenthal's 
formula as it is stated in [|1^, namely, the multiplicity of z/ as a weight of ttx is given by the 
following formula: 



where c(z/) = | |A + p| p — | |z/ + p| p, p = | J2aeA+ ^ ^u+ka is the multiplicity of u + ka 
as a weight of vr^. 

We content ourselves with an estimate. Using that p = 92+293+394 + 495+59q+697+239s, 
we compute that c(z/) = 36||0i|p. Since z/ > 0, all terms in Freudenthal's formula are 
nonnegative. Consider the roots a = |(^8 — 6*7 + Ylt=i ^i^i)^ P = \{&8 — &7 — ^i^i)^ where 
nf^j^ej = —1. Note that a + /? = 6*8 — ^7. Since (A, a) 7^ 0, we have that A — a is a weight. 
Now we have two strings of weights starting at z/, namely 



Note that in fact there are 32 different possible choices of the signs = ±1, each of which 
gives rise to a different string of weights of the type z/, u + (3. Freudenthal's formula gives 



A + p 
A — /U 



(^8 + ^6) + (^8 + ^5), 

^e,-9,) + {9s-9,). 




z/, u+i98 - 97) = X 



and 



z/, u + (3 = X — a. 



ni, 




> 



((A, ^8 - 07)mx + 32(A - «, /?)mA_„) 



1 



(2-l + 32-2-l) = — >3, 



18 



so > 4 and we are done. 



□ 



5.37 Lemma Representations numbers 25 and 31 are not of class O^. 
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Proof. Here we will use the sharper version of condition (C2) given by Lemma O , 
item (a). These representations are just 



TTx-.A-Hc ), n>4, 



1 

where o — ■•■ — cf is the vector representation. The vector representation has weights 

±6i, 1 < i < n; let {ej,e„+j : 1 < i < n} he a basis of weight vectors such that Cj (resp., 
e„+i) corresponds to the ^j-weight (resp., — 6'j-weight) weight and e(ej) = e„+i, where e is a 
invariant real structure on the representation space. 

We have that = ei A A e„+3 is a (/i = 6'i)-weight vector for tta and e(i;^) = e„+i A 



e„+3 A 63. We want to show that the condition (a) of Lemma |4.8| is violated by proving that 



the 6'2-weight vector 62 A 64 A e„+4 is not in the complex span of 

for a, /3 G A. But this follows immediately from the description of the action of a root vector 
Xa, a E A = {±{0i ±9j) : 1 < i < j < n}, on the weight vectors {cj, e„+j : 1 < z < n} under 
the vector representation, given by the following matrices: 

X-e^-Oj = En+iJ — En+j^i, 
Xe--g- = Eij — En+j^n+i^ 

where Eij denotes the 2n x 2?7,-matrix with 1 at the (i, j)-position and elsewhere. □ 

5.38 Lemma Representation number 7 is not of class O^. 

Proof. We use an argument somewhat similar to the one used in the previous lemma. 

1 1 

The representation is Tr;^ '■ A^(o — o •), where o — o • is the vector representation. Let 
ei, 62, 63, 64 = e(ei), 65 = e(e2), ee = e(e3), ey = £(67) be weight vectors of the vector 
representation corresponding to the weights 6*1, 62, O3, —9i, —62, —O3, 0, respectively, where 
e is the invariant real structure on the representation space. 

We have that = ei A 64 A ey is a (/i = 0)-weight vector for tta and e(f^) = —v^. Now 
condition (a) of Lemma is violated because the ^2-"weight vector 62 A 63 A eg is not in the 



complex span of 

for a, /3 e A. □ 
5.39 Lemma Representation number 33 is not of class . 



47 



Proof. We start with a description of the Lie algebra of G2 and of its 7- dimensional 
representation, as it is done in Let ai, 02 be the simple roots. We label the other 

positive roots as = ai + 0:2, 04 = 2ai + 02, «5 = 3ai + 02, ae = 3ai + 2a2. Choose root 
vectors Xj, Yi for ai, —ai, respectively, i = 1, 2, such that [Hi,Xi] = 2Yi, [Hi,Yi\ = —2Xi, 
where Hi = [Xi, Yi\, i = 1, 2. Next define 



-^3 = 1^1, ^2], ^3 

X4 = \[Xi, X3], 

^6 = —[^2, Yq 



-[YuY2], 
-llYuY,], 
l[Yi, F4], 
[Y2,Y,], 



and Hi = [Xi, F,] for z = 1, . . . , 6. Then [Hi, Xi] = 2Yi, [Hi, Yi] = -2Xi for i = 3, . . . , 6, and 
Xi, Yi are root vectors for ai, —ai, respectively, i = \, . . . ,6. 

The 7-dimensional representation of G2 has weights a\, —a\, 03, —as, a^, -a^, with 
respective weight vectors Vi, Wi, v^, w^, V4, w^, u, such that the action of the basis vectors 
of the Lie algebra of G2 is described in the following table: 





Hi 


H2 


Xi 


X2 




Xi 


X, 


X, 


Yi 


Y2 


Y3 


Yi 




Y, 


u 








2vi 





2vs 


2vi 








2wi 





2ws 


2wi 








V4 


f4 























V3 





-Vl 


u 


Wi 


W3 




2vi 







-V3 













u 








W3 


—Wi 







-V3 
























-Vl 


u 


-Wi 





—Wi 




—W4 





-W3 





Wi 


u 


-Vl 


-V3 




















Wi 


-2wi 


Wi 


u 








-V3 


Vi 










Wi 











W3 


W3 


-W3 





Wi 


u 


Vl 





Vi 


—Wi 


















Now we have the equation 



33 



33 © (trivial). 



from which we learn three things. First, vr^ : • 3 is a subrepresentation of the symmetric 
square of the 7-dimensional representation, so we can read it off the table above. Second, the 
multiplicity of the weights of ttx] in particular, 04 is a weight of multiplicity 2. And third, 
the trivial summand above is spanned by — + 2viWi + 2V3W3 + 2viWi, so that an invariant 
real structure on the representation space of the 7-dimensional representation is given by 
e{vi) = Wi, i = 1, 2, 3 and e{u) = —u. 

Let v^ = 2v? — SviWi + QV3W3 + QviWi, which is a real, (/i = 0)-weight vector for vr^. We 



next show that condition (a) of Lemma |4]^ is violated because the a4-weight space is not 
contained in the complex span of 



(5.40) 



'111 



XaV^,, 



XpXa{v^), 
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for a, /9 G A. In fact, 04 — can be written as a root or as a sum of two roots as follows: 



^4 — = 2ai + a2 

= ai + {ai + 02) 

= —ai + (3ai + ^2) 

= (3ai + 20:2) - («! + 0:2) 

Now applying the corresponding root vectors to we get: 

Xi{v^) = 14:{uV4 + ViV3), 

XsX^iv,) = X3(0) = 0, 
X,Y,iv,) = X,iO) = 0, 

YsXeiv,) = y3(o) = o. 



This shows that the intersection of the complex span of ( 5.4C ) with the a4-weight space has 



dimension 1; but the weight space itself has dimension 2. □ 



Proof of Lemma \5.1(\ . We eliminate the possibilities given by Table B.l and not listed in 
the table. 

3 

Let G = C„, TTx : • — • — ■■■ — • o, n > 2. Then A = 36i, ^ = —302 is a weight and 

A ± /i are neither roots nor a sum of two roots, so vr^ does not even satisfy (C2). 

1 1 

Let G = C„, Tlx : • — • — ■•• — • o, n > 3. We have 

1 1 

• • — •■• — eg) • • — = 



=0, 



1 1 



from where we deduce that the multiplicity of /i = as a weight of • — • — ••• — • o is 

2?7, — 2 > 4. Now X — is not a root and can be written as a sum of two roots in two ways, 

X-fi = {26,) + {02 - 0^) 

= (^^1 + ^2) + (^1-^3), 

and A + /i = 20i + 02 + 03 is not a root. It follows that the dimension of the intersection of 
the /i-weight space with IA'^{q'^)v\ +U^{Q'^)e{v\) is at most 2, so that condition (C^i) is not 
satisfied. 

1 

Let G = Cn, vta : • • • •=^, n > 4. Then X = 0, + 02 + 03 and this 

representation is a subrepresentation of the cubic exterior power of the vector representation. 
Also, /i = —01 is a weight of multiplicity n — 2>2. But one can check by direct computation 
that the dimension of the intersection of the /i-weight space with {q'^)v\ + U^ {Q'^)e{vx) is 1, 
so condition (Cj^i) is not satisfied. □ 
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6 The proof of the converse to a Theorem of Bott and 
Samelson 



In this section we classify variationally complete representations. The strategy is as fol- 
lows. We first exclude many irreducible representations in class from being variationally 
complete by making use of Propositions |3.7| and |3.8| . We then observe that the remaining 
irreducible representations are all orbit equivalent to isotropy representations of symmetric 
spaces, hence variationally complete by Proposition pl3| . Using Proposition |3^ , item (d), we 



see that arbitrary variationally complete representations are precisely those that are orbit 
equivalent to isotropy representations of symmetric spaces. This result can be viewed as a 
converse to Theorem |2]^ of Bott and Samelson. Finally we classify arbitrary variationally 
complete representations, mainly as a repeated application of Proposition^]^, item (b). 

Recall that the irreducible representations which according to the results in Section ^ can 
belong to class without being either the isotropy representation of a symmetric space or 
of cohomogeneity one are listed in the tables of Propositions ^A\ , |5l^ , |5.17| , |5.18| , |5.19| , f).20 



and |5.21|. For the sake of our forthcoming arguments, these representations are rearranged 



in a more systematic way in Tables C.l, C.2, C.3 and C.4 in Appendix C and in the table of 
Proposition 



6.1 Proposition The representations listed in Tables C.l, C.2 and C.3 are not variationally 
complete. 

Proof. For each representation p of the compact connected Lie group G on the real vector 
space V listed in these tables, we need to exhibit a compact symmetric space X = L/K such 
that: 

(a) K contains G as a closed subgroup; 

(b) the isotropy representation p of X restricts to p on G; 

(c) there exists p & V such that the orbits of p and p through p are the equal; 

(d) there exists q & V such that the orbits of p and p through q are different. 
It will then follow from Proposition ^.7] that p is not variationally complete. 



Let G = SO(2) X Spin(9). Then X = SO(18)/SO(2) x SO(16), p is the tensor product 
of the vector representations of SO(2), SO(16) on R^, R^^, respectively, and we take p = 
f 1 C?) f 2 € R^ ® R^^. Note that the orbits of p and p through p are equal because SO(2), 
Spin(9) and SO(16) are all transitive on unit spheres, but the orbits through a common 
regular point q are different because p is of cohomogeneity two and p is of cohomogeneity 
three. 

Let G = U(2) X Sp(n), n>2. Then X = SU(2 + 2n)/S(U(2) x U(2n)), p can be viewed 
as the realification of the complex tensor product of the vector representations of U(2) and 
SU(2?2) on C^, C^", respectively, and we take p = f i ® f 2 G C^". The orbits of p and 
p through p are equal because U(2), Sp(n) and SU(2n) are all transitive on unit spheres, 
but the orbits through a common regular point q are different because p is of cohomogeneity 
two and p is of cohomogeneity three. 
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Let G = SU(2) xSp(ri), n > 2. Then X = Sp(2 + ri)/Sp(2) xSp(?2), p is the quaternionic 
tensor product of the vector representations of Sp(2), Sp(ri) on H^, H", respectively, and 
we take p = ei ® /i + 62 ® /2 G SDh H", where {ei, 62} C and {/i, /2} C H*^ are 
orthonormal. The orbits of p and p through p are equal because both of them are 8n — 6- 
dimensional, but the orbits through a common regular point q are different because p is of 
cohomogeneity two and p is of cohomogeneity three. 

We omit the details for the representations of Tables C.2 and C.3 because they are 
analogous to the above (For instance, the reason for the existence of the common orbit 
(item (c)) is that as in the first two cases above we are dealing with tensor products of 
representations transitive on unit spheres.) Moreover, we will prove something stronger later 



in Propositions |7.5| and |7.6| , namely that these representations are not taut. □ 



6.2 Proposition The representations listed in Table C.4 are not taut. 

Proof. Let hx be a representation of the compact connected Lie group G on Va which 
is in the table. Notice that nx is of real type. Let p be a real form acting on V G Vx and 
let e be the invariant real structure on Vx. We shall go case by case and show that the slice 



representation of p at a specific point p E V is not taut, and then use Proposition |3]^ to 
conclude that p is not taut. In fact, we use p = vx + ^{vx) G V where vx is a highest weight 
vector for Tr;^. In the following, we omit the tedious and sometimes lengthy calculations 
involved and simply state the final results. 

Let G = Sp(n) x SU(6). Then the isotropy subalgebra at p is 

u(l) +su(3) +su(3) +sp(n - 1), n>2 

The dimension of the orbit through p is dim G{p) = 4n + 17, the dimension of the normal 
space at p is 36n — 17, and the complexified slice representation at p is given by: 

(trivial) 




o — o (g) o — o) (g) (trivial) 

11 1 

®{x (g) O O (g) O O © X'^ (g) O O (g) o — o) (g • •=^. 

Therefore, the slice representation contains as a summand the realification of 

1 11 
X (g O O (g O O (g • •=o. 

But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class O^. In particular, it is not taut. 

Let G = Sp(n) x Spin(12). Then the isotropy subalgebra at p is 

u(l) +su(6) +sp(n - 1), n>2 

The dimension of the orbit through p is dimG{p) = 4n + 29, the dimension of the normal 
space at p is 60n — 29, and the complexified slice representation at p is given by: 

(trivial) 
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Therefore, the shce representation contains as a summand the reahfication of 



But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class O^. In particular, it is not taut. 

Let G = Sp{n) x Sp(3). Then the isotropy subalgebra at p is 

u(l) +su(3) +sp(n - 1), n>2 

The dimension of the orbit through p is dimG'(p) = 4n + 11, the dimension of the normal 
space at p is 24n — 11, and the complexified slice representation at p is given by: 



(trivial) 

2 

^ (g)0 — o© 

2 

©(X(g)0 — o©x"^ 



© O O © X " © o — o) 



(trivial) 

2 1 

Therefore, the slice representation contains as a summand the reahfication of 



X © I 



But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class O^. In particular, it is not taut. 

Let G = Sp{n) x E7. Then the isotropy subalgebra at p is 

u(l) + C6 + 5p(n-l), n>2 

The dimension of the orbit through p is dimG(p) = An + 53, the dimension of the normal 
space at p is lOSn — 53, and the complexified slice representation at p is given by: 



(trivial) 




) © (trivial) 



Therefore, the slice representation contains as a summand the reahfication of 




=0. 



But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class C^. In particular, it is not taut. 
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Let G = Sp(l) X Spin(13). Then the isotropy subalgebra at p is 

u(l) +su(6). 

The dimension of the orbit through p is dim G{p) = 45, the dimension of the normal space 
at p is 83, and the complexified slice representation at p is given by: 

(trivial) 

1 1 
®{x^ © x^^) 

Therefore, the slice representation contains as a summand the realification of 

1 

X^ (g)' 




But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class O^. In particular, it is not taut. 

Let G = Sp(l) X Sp(2). Then the isotropy subalgebra at p is 

u(l)+u(l). 

The dimension of the orbit through p is dim G{p) = 11, the dimension of the normal space 
at p is 21, and the slice representation at p is given by the realification of: 



(trivial) 



i(x"^ ©x"^ ©x^) ©x^ 
>{x^®x~^ 
>x^ © x^. 



2x'^ © 2x^) © x^ 



Therefore, the slice representation contains as a summand the realification of a; ^ © © 
x~^ © x^ which is orbit equivalent to the circle action 



x^ © x^ 



on © R^. But the orbit of the circle action through ((1, 0), (0, 1)) is a torus-knot which is 
not taut. 

Let G = Spin(15). Then the isotropy subalgebra at p is 3u(7). The dimension of the 
orbit through p is dimG(p) = 57, the dimension of the normal space at p is 71, and the 
complexified slice representation at p is given by: 

1 1 

Therefore, the slice representation contains as a summand the realification of 
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But this is a representation of complex type with Dadok invariant 3. By Proposition it 
cannot be of class O^. In particular, it is not taut. 

Let G = Spin(17). Then the isotropy subalgebra at p is su(8). The dimension of the 
orbit through p is dim G{p) = 73, the dimension of the normal space at p is 183, and the 
complexified slice representation at p is given by: 

1 

(trivial) 



Therefore, the slice representation contains as a summand the realification of 



But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class O^. In particular, it is not taut. 

Let G = Spin(7). Then the isotropy subalgebra at p is 

u(l) +su(2) 

The dimension of the orbit through p is dim G{p) = 17, the dimension of the normal space 
at p is 31, and the slice representation at p is given by: 

(trivial) 

©(2(x^ © x'^) © (x^ © x"^)) © (trivial) 

1 

©(2(x^ © x'^) © (x^ © x'^)) © o 

2 

©((x^ © x"^) © (x^ © x"^))) © o 
Therefore, the slice representation contains as a summand the realification of 

x^ © x^ 

acting on © R^. But the orbit through ((1, 0), (0, 1)) is a torus-knot which is not taut. 
Let G = Spin(9). Then the isotropy subalgebra at p is 

u(l) +su(3) 

The dimension of the orbit through p is dimG{p) = 27, the dimension of the normal space 
at p is 101, and the complexified slice representation at p is given by: 

(trivial) 

©((x^ © x~^) © 2(x^ © x"^) © (x © x"^)) © (trivial) 

1 1 

©((x © x~^) © (x^ © x~^)) © o — o 

1 1 

©((3x © 2x° © x"^) © o — o © (3x"^ © 2x° © x^) © o — o) 

2 2 

©((x° © x"^) © o — o © (x° © x^) © o — o) 
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Therefore, the shce representation contains as a summand the reahfication of 



But this representation cannot be of class (because o — o is of real type and we use 
Lemma |5.3| , item (a)). In particular, it is not taut. 

Let G = Sp(n) x Spin(ll). Then the isotropy subalgebra at p is 

u(l) +5u(5) +sp(n - 1), n>2 

The dimension of the orbit through p is dim G{p) = An + 29 and the dimension of the normal 
space at p is 60n — 29. For n >2, the complexified slice representation at p is given by: 

(trivial) 

1 1 
{x^ (g) o — o — o — o © x~^ (g) o — o — o — o) (g) (trivial) 

(trivial) 




Therefore, the slice representation contains as a summand the reahfication of 



But this is a representation of complex type with Dadok invariant 3. By Proposition |4.9| , it 
cannot be of class O^. In particular, it is not taut. 

For n = 1 we need a different argument. In this case the connected component K of the 
isotropy group at p is locally isomorphic to U(5), and the slice representation at p contains 
as a summand the reahfication of 

1 

Denote by Vi, V2 the respective representation spaces of the above summands. Then Vi = 
and V2 = A^C^. Choose an orthonormal basis {ei, . . . , 65} for C^. Let q2 = aei A 62 + be^ A 
64 G V2 be a regular point, where a, b are distinct positive real numbers. We have that 
the isotropy Kg^ is locally isomorphic to SU(2) x SU(2) x U(l) sitting diagonally in K. 
Let gi = Ci + 63 G Vi. Now the orbit Kq^{qi) is diffeomorphic to 5*^ x S^^ whereas the 
orbit K{qi,q2) is diffeomorphic to SU(5). It is known that the third Betti number of a 



compact connected simple Lie group is one. It follows from Proposition |3.11| that the slice 



representation at p is not taut. □ 

6.3 Proposition The following is a list of pairs of irreducible representations p, p' of com- 
pact connected Lie groups G, G' such that p' is the isotropy representation of a symmetric 
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space, G is a subgroup of G' and p = p'\G is orbit equivalent to p' . 



G and G' 



SU(ra) X SU(m), n ^ m 
and 

X SU(n) X SU(m) 



SU(n), n odd, and 
U(n) 



Spin(lO) and 



SO(2) X Spin(lO) 



SO(2) X Spin(7) and 
SO(2) X SO(8) 



SO(2) X G2 and 
SO(2) X SO(7) 



SO(3) X Spm(7) and 
SO(3) X SO(8) 



p and p' 



— O (g) o o) 

1 

© (o o®c^ 



and 



-o®x® o — 



-o 



(o o (g) x^^ ® (y— 



-o) 



— 0©0 — •■■ — O — O and 



—O) © {x-^ 




{x®x-^) © 



ix © a;"M © o — o 




and 



X © X 



{x ® X ^)©«^33 and 



[X © x~ 



o — o 




Proof. The first three cases in the table were treated in Lemma |5!^ , item (a). It remains 
to analyze the other three cases. It suffices to show the existence of a common principal orbit 
by Lemma |3.6| . 

Let G = SO(2) X Spin(7). Then p is the tensor product of the vector representation of 
SO(2) on and the spin representation of Spin(7) on R^. Let p = aiCi © /i + 0262 © /2 G 
R^ © R^, where ei, 62 € R^ are orthonormal, /i, /2 G are orthonormal, and ai, a2 are 
distinct positive numbers. Then the orbit G'p is principal and of dimension 14. Since the 
isotropy subalgebra of G at p is easily seen to be su(3), it follows that Gp is also an orbit 
of dimension 14. Hence, it coincides with G'p. It follows from Lemma ^]6| that p and p' are 
orbit equivalent. 

Similarly, one sees that in the last two cases in the table p has respectively an orbit of 
dimension 12, 21, and therefore is orbit equivalent to p'. □ 



56 



6.4 Theorem (Converse to a Theorem of Bott and Samelson) A variationally com- 
plete representation of a compact connected Lie group is orbit equivalent to the isotropy 
representation of a symmetric space. 

Proof. We first assume that the given representation is irreducible. A variationaUy 
complete, irreducible representation p of a compact connected Lie group G is of class O^. 
If p is not the isotropy representation of one of the irreducible symmetric spaces and has 



cohomogeneity greater than one, we have obtained all possibilities for it in Propositions 5.1 



5^ , p.l7| , |5.18| , p.l9| , |5.20| and |5.21| , and these are listed in the table of Proposition |6]^ and 



in Tables C.l, C.2, C.3 and C.4 in Appendix C. Now Propositions |0| and |6.2| eliminate 
all possibilities for p, except those listed in the table of Proposition |6.3| . But the same 
proposition asserts that p is orbit equivalent to another representation p' which is the isotropy 
representation of a symmetric space. 

Now assume that the representation is reducible. Each of its irreducible factors is varia- 
tionally complete by part (a) of Proposition and hence orbit equivalent to the isotropy 
representation of an irreducible symmetric space by what we have just proved. Part (d) 
of the same proposition now implies that the reducible representation is orbit equivalent to 
the isotropy representation of the symmetric space that is the product of the irreducible 
symmetric spaces corresponding to the irreducible summands of the representation. □ 

6.5 Corollary A variationally complete representation of a compact connected Lie group is 
polar. 



The first paragraph in the proof of Theorem |6.4| combined with Theorem to be proved 
in Section | also implies: 

6.6 Theorem (Compare ||T5|| ) Let p : G ^ a variationally complete irreducible 

representation of a compact connected Lie group G. Then there are two cases: 

(a) The image p{G) is the maximal compact connected Lie group with its orbits. In this 
case p is the isotropy representation of an irreducible symmetric space. 

(b) The image p{G) is not the maximal compact connected Lie group with its orbits. In 
this case we have: 

(i) If p has cohomogeneity greater than one, then there exists a larger group G' D G 
and an isotropy representation of an irreducible symmetric space p' : G' ^ O(^) 
which is an extension of p. Moreover, the possibilities for the pair {p,p') are listed 



in the table of Proposition 6. S 



(a) If p has cohomogeneity one, then it is listed in the tables of Theorem^ 

In order to complete the classification of variationally complete representations, we finally 
discuss the reducible case and state a result in Theorem ^]8| which, in view of Theorem |6.4| , 
is equivalent to a result of Bergmann in First note that there are some trivial ways to 
produce variationally complete reducible representations from known variationally complete 
representations. 

Let p : G — > O(V^) be a representation of a compact connected Lie group G. 
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(a) Assume G = G' x K where G', K are compact connected Lie groups. We say that K is 
nonessential for p if the restriction p\G' is orbit equivalent to the original representation 
p. Otherwise, we say that K is essential for p. 

(b) Assume p reducible, namely p = pi ® P2 where pi : G ^ and dimV^ > 
{i = 1, 2). We say that p is splitting if we can write G = Gi x G2 for compact 
connected Lie groups Gj (i = 1, 2) such that p is equivalent to the outer direct sum 
representation pi|Gi©p2|G'2 : Gi x G2 ^ 0(Vi © V2). We say that p is almost splitting 
if we can write G = Gi x G2 x K for compact connected Lie groups K, Gi {i = 1, 2) 
such that K is nonessential at least for one of pi|Gi x K, P2IG2 x K and p\Gi x G2 is 
equivalent to the outer direct sum representation pi|Gi©p2|G'2. 

It is immediate from these definitions that a representation with a trivial summand is auto- 
matically splitting, and that splitting implies almost splitting. We also have: 

6.7 Proposition An almost splitting representation p = pi © P2 is variationally complete if 
and only if each one of its summands pi is variationally complet^. 

Proof. One direction follows from Proposition |3.4| , item (a); we prove that the summands 
being variationally complete implies the same for the sum. We may assume that K is 
nonessential for pi|Gi x K. Then pi is orbit equivalent to pi|Gi and p2 is orbit equivalent 
to p2|G'2 X K, so pi|Gi and P2|G'2 x K are variationally complete. Now p is orbit equivalent 
to pi|G'i©p2|G2 X K and hence variationally complete, because the outer direct sum of two 
variationally complete representations is also variationally complete, as is easy to see. □ 

We are left with the classification of variationally complete reducible representations 
which are not almost splitting. This is given in the following theorem. 

6.8 Theorem (Compare Let p be a variationally complete reducible representation of 
a compact connected Lie group G which is not almost splitting. Then p is one of the following 

^Note that the isotropy representations of reducible symmetric spaces are sphtting. 
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orthogonal representations: 



G 


P 


SU(4) 

Spin(7) 
SO(2) X Spm(7) 

Spin(8) 


1 11 
1 11 
1 1 

O 0=» © O 0=9 


Spin(8) 




SO(2) X Spin(8) 

\ / Mr \ / 


(x©x"^)©o 0^ ©o — 


SO(2) X Spin(8) 


(x©x"^)©o — ©o — 


SO(3) X Spin(8) 


o© o — © o — 


SO(3) X Spin(8) 


o© o — © o — 



Sketch of a proof. Bergmann proves her theorem in using Theorem |6.6| as it is stated 
in [|15|. We sketch here an argument in a similar vein, but divided into cases in a different way. 
Since summands of almost faithful reducible representations need not be almost faithful, this 
is the only place in this paper where we have to exercise some care in distinguishing between 
a group G and its image p(G) under a representation p : G ^ 0(^)- 

G IS A SIMPLE GROUP. Let p = pi © p2 be a decomposition where pi is irreducible. 
Then pi, p2 are variationally complete. Denote by Vi, V2 the representation spaces and 
by Hi, H2 the connected components of the principal isotropy subgroups. We have that 
pi is an almost faithful variationally complete irreducible representation of a simple group, 
which is orbit equivalent to the restriction pi|i?2, where H2 is a proper subgroup of G; see 
Proposition |3.4| , item (b). Since pi{H2) C 0(Vi) is not the maximal compact connected 
group with its orbits, pi : H2 0(Vi) is not the isotropy representation of a symmetric 
space. It follows from Theorem |6.6| that pi has cohomogeneity one, because the larger of 



the two groups for each pair in the table of Proposition is never a simple group. Since 
pi is an arbitrary irreducible summand of p, we conclude that every irreducible summand 
of p must have cohomogeneity one. Moreover, we know from Proposition p.4| item (c), that 



there cannot be two equivalent summands. The only simple groups admitting at least two 
inequivalent cohomogeneity one representations are Spin(n) for n G {3,5,6,7,8,9}. Using 



59 



the fact that p2 must be orbit equivalent to p2\Hi we exclude the groups Spin (3), Spin (5), 
Spin (9) and get the four examples corresponding to simple groups in the table. 

G IS NOT SEMISIMPLE. Let pi be an irreducible summand of p such that pi{G) is locally 
isomorphic to 5"^ x G', where G' is a nontrivial semisimple factor of G. Let p2 be another 
irreducible summand of p with the connected component of the principal isotropy group 
being H2. If Pi(-f^2) = Pi{G) for all such p2, then p is splitting. So we assume there exists 
such a P2 with pi(i^2) £ Pi{G)- Now we can view pi as an almost faithful variationally 
complete irreducible representation of 5*^ x G\ which is orbit equivalent to the restriction 
Pi\H'2, where H2 is a compact connected proper subgroup of x G' and pii^H'^) = pi{H2). 
It follows from Theorem |6.6| , item (b), that the possible local isomorphism classes of H2, 
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X G' and the representations pi|i?2> Pi\^^ ^ ^' given as follows: 





and X G' 


pi\H^ and pil^^xG" 


1 


SO (2) X G2 and 


1 

{x © x~^) (8) ♦ and 




SO(2) X SO(7) 


1 

{x © x"^) (g) — c^=» 


2 


SO (2) X Spin(7) and 


1 

{x © (g) — C^=« and 




SO(2) X SO(8) 


(a;©a;"^) O — 


3 


Spin(lO) and 




— — and 




SO(2) X Spin(lO) 


(x © x~^) — — 


4 


SU(n), n odd, and 


1 1 
— © — and 




X SU(n) 


1 1 
{x (g) — 0) © {x-^ (g) — 0) 


5 


SU(n) X SU(m), n ^ m 
and 


1 1 

(0 (g) 0) 

1 1 
© (0 (g) 0) and 




X SU(n) X SU(m) 


1 1 

(0 (g) x (g) 0) 

1 1 
© (0 (g) a;-^ (g) 0) 


6 


Sp(n) and 


1 

• 0=0 and 




5"^ X SU(2n) 


1 1 

X (go 0© (g)0 


7 


SU(n) and 


1 1 

© and 




X SU(n) 


1 1 
X (g © x~^ (g 


8 


X Sp(n) and 


1 

{x © x~^) (g) • — ■•• — • and 




X SU(2n) 


1 1 
X (g)0 0© (g)0 


9 


Sp(n) and 


1 




X Sp(n) 


1 



In the following we examine different possibilities for p2. Recall that a variationally 
complete representation does not admit two equivalent nontrivial irreducible summands. 

(i) P2 is as in case 1. Here p2 is trivial on SO (2), so we can view p2 as an almost faithful 
representation Spin(7) x K —>■ 0(V2) {K is a compact connected Lie group) which 
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is variationally complete, irreducible and such that the projection of the connected 

component of its principal isotropy subgroup in Spin(7) is G2. The only possibility 

1 

for P2 is O — O K = {1}. Any other irreducible summand ps of p either would 
satisfy pii^H^) = pi{G) (where H-^ is the connected component of the principal isotropy 
subgroup of Pa) or would have to be in the same case as p2. The first possibility 
would make p splitting and the second possibility is excluded because p cannot have 
two equivalent irreducible summands. It follows that p has exactly two irreducible 

summands and \t \s {x ® x~^) ® O — 0==» © O — 0==». 

P2 is as in case 2. Here p2 is trivial on SO(2), so we can view p2 as an almost faithful 
representation Spin(8) x — > 0(V2) {K is a compact connected Lie group) which 
is variationally complete, irreducible and such that the projection of the connected 
component of its principal isotropy subgroup in Spin(8) is Spin(7). Here Spin(7) is in 
the conjugacy class such that the vector representation of Spin(8) restricts to the spin 
representation of Spin(7). The only possibilities for p2 arc the half-spin representations 
of Spin(8), K = {!}. Similar considerations to those in (i) show that p must have 
exactly two irreducible summands and that it is either one of 

{x®x~^)(^o — ©o — ' 

P2 is as in one of the cases 3, 4, 5 or 9. Here is not essential for pi and G' C ker p2. 
Also, for any other irreducible summand p^ either pi{H-i) = pi{G) (where i/3 is the 
connected component of the principal isotropy subgroup of ps) or G' C ker p^. It follows 
that p is almost splitting. 

P2 is as in case 6. This case cannot occur, for otherwise p2 : U(2n) x K ^ 0(V2) {K a 
compact connected Lie group) is an almost faithful variationally complete irreducible 
representation such that the projection of the connected component of its principal 
isotropy subgroup in U(2n) is Sp(n). But such a p2 does not exist. 

P2 is as in case 7. Here is not essential for pi and G' C kerp2. Also, for any other 
irreducible summand p3 we have either one of the following: p\{H^) — P\{G) (where 
is the connected component of the principal isotropy subgroup of ps); or G' C kerpa; 
or p3 is as in case 8. In the first two possibilities p is almost splitting and in the third 
one we go to the next case. 

P2 is as in case 8. Here p2 is trivial on S^, so we can view p2 as an almost faithful 
representation SU(2n) x K 0(V2) {K is a compact connected Lie group) which 
is variationally complete, irreducible and such that the projection of the connected 

component of its principal isotropy subgroup in SU(2n) is Sp(n). The only possibility 

1 

for p2 is o — o — o, K = {1}, n = 2. Ifpsisa third irreducible summand of p then 
either pi{H^) = Pi{G) (where is the connected component of the principal isotropy 
subgroup of Pa) or pi(i?3) must be as in case 7, therefore pa is trivial on SU(4). So if 
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there is any irreducible summand besides pi and p2 then p is almost splitting. Therefore 
there are no other irreducible summands and p is 



1 



[x ® o O O © X ^ ® I 



G IS SEMISIMPLE BUT NOT SIMPLE. Let pi be an irreducible summand of p such that 
Pi{G) is locally isomorphic to G", a semisimple but not simple factor of G. Let p2 be another 
irreducible summand of p with connected component of the principal isotropy group H2. If 
Pi{H2) = pi{G) for all such p2, then p is splitting. So we assume there exists such a p2 with 
Pi{H2) ^ Pi{G)- Now we can view pi as an almost faithful variationally complete irreducible 
representation of G', which is orbit equivalent to the restriction pilH!^, where H2 is a compact 
connected proper subgroup of G' and piiH!^) = pi{H2). It follows from Theorem |6.6|, item (b), 
that the possible local isomorphism classes of H2, G' and the representations pi\H2, pi\G' 
are given as follows: 





H!^ and G' 


Pi\H2 and pi\G' 


10 


Sp(n) and 


1 




Sp(l) X Sp(n) 


1 1 


11 


SO(3) X Spin(7) and 


2 1 




SO(3) X SO(8) 


2 1 P 
00 — c/ 



If P2 is an in case 10, then p is almost splitting by the same argument as in item (iii). If p2 
is as in case 11, then by an argument similar to that in item (ii) we show that p must be 
either one of 




7 Taut representations 

The main result of this section is the following classification theorem. 

7.1 Theorem A taut irreducible representation p of a compact connected Lie group G is 
either orbit equivalent to the isotropy representation of a symmetric space or it is one of the 



63 



following orthogonal representations (n > 2): 



G 


P 


SO(2) X Spin(9) 
U(2) X Sp{n) 
SU(2) X Sp(n) 


1 

{x © x-i) (g) o O (>=• 

1 1 

3 1 

O (g) • 



7.2 Corollary ^4 tafii irreducible representation of a compact connected simple Lie group is 
orbit equivalent to the isotropy representation of a symmetric space. 



7.3 Remark According to see also the representations given in the table of The- 
orem |7[l| are exactly the irreducible representations of compact connected Lie groups which 



have cohomogeneity three and are not orbit equivalent to the isotropy representation of a 
symmetric space. 



It follows from the main results of Section ^ that Theorem |7[T| is equivalent to the follow- 



mg: 

7.4 Theorem A taut irreducible representation p of a compact connected Lie group G is 
one of the following: 

(a) The isotropy representation of an irreducible symmetric space (listed in Tables 8.11.2 
and 8.11.5 of g^). 



(b) A cohomogeneity one representation not listed in (a) (cf. Theorem\ 

(c) A representation orbit equivalent to the isotropy representation of an irreducible sym- 



metric space and not listed in (a) or (b) (cf. Proposition "UTS) 
(d) One of the three exceptional cases given in Theorem \7.1\ . 



Proof of Theorems ITJI andVTJ. A taut irreducible representation p of a compact connected 
Lie group G is of class O^. If p is not orbit equivalent to the isotropy representation of one of 
the irreducible symmetric spaces, we have obtained all possibilities for it, and these are listed 



in Tables C.l, C.2, C.3 and C.4 in Appendix C. Now we already know from Proposition |6T2 
that the representations in Table C.4 are not taut. Therefore the assertion follows from 

□ 



Propositions [TS] , |^6| and |7.12| below. 
We start by considering Table C.3. 



7.5 Proposition The representations listed in Table C.3 are not taut. 
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Proof. Let G = G2 x Spin(7). Then p is the tensor product of the 7- dimensional 
representation of G2 on R'' and the spin representation of Spin(7) on R®. Let p = fi ®f2 G 
R'^ ® R^. The isotropy subalgebra at p is su(3) + 92- The dimension of the orbit through p is 
dimG'(p) = 13, the dimension of the normal space at p is 43, and the slice representation at 
p minus the trivial component Rp is given by the tensor product of the vector representation 
of su(3) on R^ and the 7-dimensional representation of Q2 on H^. Since this representation 
is of complex type and Dadok invariant 3, it follows from Proposition |4.9| that it cannot be 
of class O^. In particular, it is not taut. Now Proposition p.8| implies that p is not taut. 

Let G = Spin(7) x Spin(9). Then p is the tensor product of the spin representation of 
Spin(7) on R^ and the spin representation of Spin(9) on R^^. Let p = f 1 ® f 2 G R* ® R^^. 
The isotropy subalgebra at p is 92 + spin(7). The dimension of the orbit through p is 
dimG(p) = 22, the dimension of the normal space at p is 106, and the slice representation at 
p minus the trivial component Hp is given by the real tensor product of the 7-dimensional 
representation of Q2 and the vector representation of 5pin(7) plus the real tensor product 
of the 7-dimensional representation of 02 and the spin representation of spin(7). Since we 
already know that the tensor product of the 7-dimensional representation of 02 and the spin 
representation of spin(7) is not taut (see the last paragraph), it follows from Proposition |3lS 
that p is not taut either. 

Let G = G2 X Spin(9). Then p is the tensor product of the 7-dimensional representation 
of G2 on R^ and the spin representation of Spin (9) on R^^. Let p = f 1 ® f 2 G R^ ® R^^. 
The isotropy subalgebra at p is 5u(3) + 5pin(7). The dimension of the orbit through p is 
dimG(p) = 21, the dimension of the normal space at p is 91, and the slice representation 
at p minus the trivial component Hp is given by the real tensor product of the vector 
representations of su(3) and spin(7) plus the real tensor product of the vector representation 
of su(3) and the spin representation of spin(7). Since the second summand is a representation 
of complex type and Dadok invariant 3, it follows from Proposition |4.9| that it cannot be of 
class O^. In particular, it is not taut. Now Proposition implies that p is not taut. 

ConsideiQ G = Sp(l)-Sp(n) X G2, > 2. Let ?:„ be a real form of 0®0 — O — ■•• — O. Then 
Tn can be realized as the representation of Sp(l) ■ Sp(?2) on R"^"- given by r„(g, A)x = Axq~^, 
where g e Sp(l), A e Sp(n), x G H" and we identify H" ^ R"^". Now p is the real tensor 
product of Tn and the 7-dimensional representation of G2. Let p = f 1 (S> f 2 G R*^" ® R^. 
The isotropy subalgebra at p is isomorphic to sp(l) + sp{n — 1) + 5u(3). The dimension 
of the orbit through p is dimG(p) = 4n + 5, the dimension of the normal space at p is 
24n — 5, and the slice representation at p minus the trivial component Hp is given by the real 
tensor product of r„_i and the vector representation of su(3) plus the real tensor product 
of the adjoint representation of sp(l) and the the vector representation of su(3). Since the 
second summand is a representation of complex type and Dadok invariant 3, it follows from 
Proposition [4.9| that it cannot be of class O^. In particular, it is not taut. Now Proposition |3!S 
implies that p is not taut. 

Consider G = Sp(l) ■ Sp(n) x Spin(7), n > 2. Let above. Now p is the tensor 

product of Tn and the spin representation of Spin(7). Let J9 = f 1 ® f 2 G R*^"" ® R^. The 
isotropy subalgebra at p is isomorphic to sp(l) + sp{n — 1) + Q2- The dimension of the 
orbit through p is dim G{p) = 4?t, + 6, the dimension of the normal space at p is 28n — 6, 

^We use the convention that G ■ H refers to a quotient of G x iJ by a finite central subgroup. 
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and the slice representation at p minus the trivial component is given by the real tensor 
product of r„_i and the 7-dimensional representation of 02 plus the real tensor product of 
the adjoint representation of sp(l) and the the 7-dimensional representation of Q2- Since the 
first summand is not a taut representation by the above (for = 2 we refer to the case of 
G = SO(4) X G2 which will be dealt with in Proposition |7.6| ), it follows from Proposition |3^ 
that p is not taut either. 

Let G = Spin(7) x Spin(7). Then p is the tensor product of the spin representations 
of each of the factors on R®. Let p = Vi ® V2 & R^ (S> R*^. The isotropy subalgebra at p is 
02 + 02- The dimension of the orbit through p is dimG(p) = 14, the dimension of the normal 
space at p is 50, and the slice representation at p minus the trivial component Hp is given 
by the real tensor product of the 7-dimensional representations of each of the factors. Since 



this representation is not taut (see next case), it follows from Proposition |3.8| that p is not 
taut either. 

Consider G = G2 x G2. Then p is the tensor product of the 7-dimensional representations 
of each of the factors on R^. Let {ei,... ,67} be an orthonormal basis of R^ and take 
p = ei ® ei + . . . + 67 (g) 67. Then the isotropy subalgebra at p is 

0p = {(X,X):XG02} 

which is isomorphic to 02- Now the restriction of p to Qp is a real form of 

11 11 

02 : ® i^^y = ® 

acting on R^^, and R"^^ = TpG{p) © NpG{p) is a 0p-invariant decomposition. We have the 
following decomposition into irreducible components: 



02:®^(^^) = 5^(«))©A^(«3) 

2 11 
= (•^o © (trivial) ) © (^^ © •^o) 



Since TpG{p) is 14-dimensional, it follows that the 27-dimensional real representation • o 
must be a component of the slice representation at p. But we saw in Lemma |5.39| that it is 
not of class O^. Now Proposition |3.8| implies that p is not taut. □ 

7.6 Proposition The representations listed in Table C.2 are not taut. 

Proof. Let G = SO(m) x G2, m > 4. Then p is the tensor product of the vector 
representation of SO(m) on R*" and the 7-dimensional representation of G2 on R''. Let 
p = fi©f2 €R'"©R''. The isotropy subalgebra at p is so (m — 1) +su(3). The dimension of 
the orbit through p is dim G{p) = m + 5, the dimension of the normal space at p is 6m — 5, and 
the slice representation at p minus the trivial component Hp is given by the tensor product 
of the vector representations of so(m — 1) and su(3) on R'^~^ and R^, respectively. Since this 



is a representation of complex type and Dadok invariant 3, it follows from Proposition ^ 
that it cannot be of class O^. In particular, it is not taut. Now Proposition p.8| implies that 
p is not taut. 

The proof for G = SO (3) x G2 will be done in Lemma |7.8| below, since this case is more 
involved. 
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Let G = SO(m) x Spin(7), m > 4. Then p is the tensor product of the vector represen- 
tation of SO(m) on R™ and the spin representation of Spin(7) on R®. Let p = f i ® f 2 G 
R™ ® R^. The isotropy subalgebra at p is so{m — 1) +02- The dimension of the orbit 
through p is dim G{p) = m + 6, the dimension of the normal space at p is 7m — 6, and the 
shce representation at p minus the trivial component Hp is given by the real tensor product 
of the vector representation of so{m — 1) and the 7-dimensional representation of 02- Since 
this representation is not taut by the above, it follows from Proposition that p is not taut 
either. 

Let G = SO(m) x Spin(9), m > 4. Then p is the tensor product of the vector represen- 
tation of SO(m) on R*" and the spin representation of Spin(9) on R^^. Let p = f 1 ® f 2 G 
R"^ ® R^^. The isotropy subalgebra at p is so(m — 1) + spin(7). The dimension of the orbit 
through p is dim G{p) = m + 14, the dimension of the normal space at p is 15m — 14, and the 
slice representation at p minus the trivial component Hp is given by the real tensor product 
of the vector representations of so(m — 1) and spin(7) plus the real tensor product of the 
vector representation of so{m — 1) and the spin representation of spin(7). If m > 5 then the 
second summand is not a taut representation by the above and it follows from Proposition RT^ 



that p is not taut either. In the case m = 4 we give a special argument. Here we have that 
the connected component of the isotropy subgroup at p is K = SO (3) x Spin(7). Let ui, 
i>2 be respectively the irreducible summands of dimensions 21, 24 of the slice representation 
at p, and denote with Vi, V2 the representation spaces. Note that both z/i and 1^2 are taut 
representations of K, since ui is the isotropy representation of a real Grassmann manifold 
and z/2 is orbit equivalent to the isotropy representation of a real Grassmann manifold. Let 
qi G be a regular point. Then the isotropy group Kg^ is isomorphic to x Spin(4). We 
can choose q2 G V2 so that the isotropy {Kq^)q^ is contained in {1} x Spin(4). Hence the 
orbit Kq^[q2) is disconnected. Since the orbit K{qi, ^2) is connected. Proposition p.ll| implies 
that the slice representation at p is not taut. Now Proposition p.8| yields that p is not taut 
either. 

We postpone the proof for the case SO (3) x Spin(9) to Lemma fTT] below. 

Let G = SU(n) x Sp(m), tt, > 3 and m > 2. Then p is the realification of the tensor 
product of the vector representations of SU(ri) on C" and of Sp(m) on C^™. Consider 
complex bases {ei, ... , e„} for C" and {/i, ... , /„, fm+i, • • • , /2m} for C^™ where fm+j = efj, 
e the quaternionic structure on C^"*. Let p = ei® f\. Then the isotropy K = Gpis isomorphic 
to To ■ SU(n — 1) X Sp(m — 1), where Tq is the circle group generated by 



</.o(t) = (diag(e("-i)**,e-^*,...,e-^*) 

diag(e-("-^)'*, 1, . . . , 1; e^""^)'*, 1, . . . , 1)) G SU(n) x Sp(m), 

and the slice representation Vp minus the trivial component corresponding to the radial 
direction Rp decomposes into irreducible components as vi ® ^2-, where vi restricted to 
SU(n — 1) X Sp(m — 1) is the realification of the complex tensor product of the vector repre- 
sentations of the factors and z/2 is the realification of the vector representation on SU(r2 — 1) 
and it is trivial on Sp(m — 1). Denote the representation spaces by Vi, V2. Then Vi is 
spanned by Cq, ® fp and i{ea ® //?), for2<a<n, 2</?<morm + 2</?< 2m, and V2 
is spanned by Cq, ® /m+i and i{ea ® fm+i), for 2 < a < n. Let = 62 ® /2 G V^i. Then the 
isotropy Ki = Kq^ is isomorphic to Ti ■ T2 ■ SU(n — 2) x Sp(m — 2), where Ti, T2 are circle 
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groups respectively generated by 



ipi{t) = (diag(l,e("-2)**,e-^*, 



1,... ,1)) e SU(n) X Sp(m) 



and 



Lp2it) = (diag(e 
diag(e^ 



(n—l)it „—it 

-{n—l)it it 
1 fc. , 



,l;e 



(n— 1 
1 1 -Li 



1)) G SU(n) X Sp(m) 



(Here we have used the hypothesis n > 3 and m > 2). Finally let q2 = e2 ® fm+i G V2. 
Then the isotropy K2 = {Kgjg^ is isomorphic to T3 ■ SU(n — 2) x Sp(m — 2), where T3 is the 
circle subgroup of Ti ■ T2 generated by (p3{t) = ipi{—t)ip2{t). It is easy to see that K1/K2 is 
diffeomorphic to S*^, whereas K/K2 is simply-connected. Therefore Proposition 3.11| implies 
that z/p is not taut. Now Proposition yields that p is not taut either. 

The case where G = U(n) x Sp(m), n > 3 and m > 2, is completely analogous to the 
previous case. 

Let G = SO(m) x Sp(l) • Sp(r;,), m > 3 and n > 2. Then p is the tensor product of 
the vector representation of SO(m) on R"* and r„, where r„ is a representation on R^" as in 



Proposition |7 
basis 



We choose an orthonormal basis {ei, . . . , Cm} for R"* and an orthonormal 



{/l, • • • ,fn, fli, ■■■ , fni, flj, ■■■ , fnj, flk, ... , fnk} 

for R^". Let p = ei ® fi. The connected component of the isotropy subgroup K 
isomorphic to SO(m — 1) x Sp(l)' ■ Sp(n — 1), where 



Gl is 



Sp(l)' 



r / 



v 



v 



:gGSp(l) 



> C Sp(l) ■ Sp(n). 



The slice representation at p minus the trivial component Rp decomposes as z/i ©z/2, where vi 
is the real tensor product of the vector representation of SO(m — 1) and r„_i, and z/2 is the 
real tensor product of the vector representation of SO(m — 1) and the adjoint representation 
of Sp(l) (z/2 is trivial on Sp(n — 1)). Note that z/2 is the isotropy representation of a real 
Grassmann manifold. Let Vi, V2 denote the representation spaces of z/i, z/2. Then Vi is 
spanned by Cq, ® fp, Cq, ® fpi, Cq, f/sj, Cq, ® fpk, for 2 < a < m, 2 < j3 < n, and V2 is 
spanned by Cq, ® fii, Cq, ® fij, Cq, ® fik, for 2 < a < m. First consider the case m > 4. 
Let g2 G V2 be a regular point, say q2 = ae2 <S) fii + be^ ^ fij + ce^ fik, where a, b, 
c are pairwise distinct positive real numbers. Then the isotropy group Kg^ is isomorphic 
to SO(m — 4) X Q ■ Sp{n — 1), where Q is the quaternion subgroup {±1, ±z, ±j, ±A;} of 
Sp(l). Consider the restriction of z/i to Kg^. We can choose qi G Vi so that the orbit 
Kg^{qi) is disconnected, say qi = ae2 ® /2 + be^ ® f2h where a, b are distinct positive real 
numbers. In fact, if {A,q,B) G SO(m — 4) x Q ■ Sp(n — 1) is in the isotropy of gi, then 
we must have Bf2q~^ = f2 and Bf2iq^^ = /2^, which implies that Bf2q~^i = Bf2iq~^ 
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and then q~^i = iq~^, so that q = ±1 or g = ±i. In particular, no element of the form 
{A,j, B) G SO(m — 4) X Q ■ Sp(n — 1) is in the isotropy of gi, which implies that the orbit 
Kq^{qi) is disconnected. Since the orbit K{qi,q2) is connected, Proposition |3.11| implies that 
the slice representation at p is not taut. Now Proposition p.8| yields that p is not taut either. 
Finally we analyse the case m = 3. Then K = SO(2) x Sp(l)'-Sp(?7. — 1). Consider a regular 
point q2 G V2, say q2 = ae2 (8> fii + be-^ ® fij, where a, b are distinct positive real numbers. 
Then we have that the isotropy K^^ is isomorphic to Z4 x Sp{n — 1). Here Z4 is generated 
by {—l,k), where —1 is minus the identity matrix in SO(2) and k G Sp(l)'. Consider the 
restriction of z/i to Kg^. We can choose qi G Vi as above such that its isotropy does not 
contain elements of the form (—1, k, B), for B G Sp(n — 1). It follows that the orbit Kq^{qi) 
is disconnected and the proof follows as above. □ 

7.7 Lemma The tensor product p of the vector representation o/SO(3) on and the spin 
representation of Spin(9) on R^^ is not taut. 

Proof. Let p = f 1 (S> f 2 G R'^ ® R^^. Then the connected component of the isotropy 
group K = Gp is isomorphic to SO(2) x Spin(7). The normal space at p to the orbit G{p) 
decomposes as Hp © Vi © V2, where Vi = R^ ® R'', V2 = R^ © R^ are the representation 
spaces for the irreducible components ui, V2 of the slice representation at p. Here vi is 
the tensor product of the vector representations and z/2 is the tensor product of the vector 
representation of SO(2) and the spin representation of Spin(7). Now let g = f 3 © f 4 G V2. 
Then the connected component of the isotropy group is isomorphic to G2 and the slice 
representation at q of the representation of K on Vi©V2 is a representation of G2 on R^^ which 
decomposes as a trivial component Rg plus three copies of the 7-dimensional representation 
of G2. Next we shall show that the sum /i = /ii © /i2 © A'-s of three copies of 7-dimensional 
representation of if = G2 is not taut. It then follows from Proposition p.8| that p is not taut 
either. 

In fact, let Wi be the representation space of pi. We can select Wi G Wi such that 
the isotropy groups H^^, (if^J^„2, {{Hw^)yj,^)u,.^ are respectively SU(3), SU(2), {1}. Since 
H{wi_, W2,wz) = ((-f^iui)«)2)«'3 5 W6 get that the orbit H{wi,W2,w^) is diffeomorphic to G2. Now 
if this orbit is taut. Proposition |3.11| implies that it must have the product homology of the 
orbits through i^i, W2 and 1^3, namely, S*^, and S^. But it is well known that G2 has the 
product homology of S^^ and (this also follows from Satz 1 in [^). Thus, p is not taut. 

□ 

7.8 Lemma The tensor product p of the vector representation of SO (3) on R'^ and the 
7-dimensional representation of G2 is not taut. 



Proof. We shall use the reduction technique described in Subsection p.3| . Note that 
G = SO (3) X G2 is a closed subgroup of G = SO (3) x SO (7) and p is the restriction of a 
representation p of G which is the isotropy representation of a real Grassmann manifold. 

We view the Cayley numbers O = H © eH, so that {l,e,i,j,k,ei,ej,ek} is a real or- 
thonormal basis for O. The Lie group G2 is the automorphism group of O with respect to 
its nonassociative algebra structure. We next compute the space of orbits of G acting by p 
on = R'^ © R^. Take {e, i,j, k, ei, ej, ek} as a basis for R'' and take an orthonormal basis 
{fi, f2, fs} for R'^. Recall that the subspace spanned by {i,j,k,ei,ej,ek} has a complex 
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structure given by left multiplication by e. Since all these representations are self- adjoint, it 
is equivalent to consider p acting on R^* ® which we may identify with the space of 3 x 7 
real matrices. 
Let 

/lOOOOOOX 
Pi = 0000000 , 

yooooooo/ 

Then the orbit G{pi) is 8-dimensional and its normal space at pi intersects all orbits. So, 
given an arbitrary orbit M, it contains a point of the form 

a 
***** * 
***** * 

for some a > 0, for we can multiply on the left by 

/ -1 \ 
-10 

V i; 

to change the sign of a, if necessary, and remain in the orbit M. Now the connected compo- 
nent of the isotropy subgroup at pi, Ki — SO(2) x SU(3), acts on the normal space to 
by fixing the radial direction Rpi and by the tensor product of the vector representations of 
SO(2) and SU(3) on the orthogonal complement R^^. The point 

/OOOOOOOX 
P2 = 0100000 , 
\0000000/ 

is in R^^ and has a 6-dimensional Xi-orbit. So M has a representative of the form 

aOOOOOOX 
& , 
00*****/ 

for some 6 > 0, since we can also multiply on the left by 

/I \ 

0-1 , 

Vo -I I 

if necessary, and remain in the same orbit. Finally, the connected component of the isotropy 
subgroup of Ki at p2 is SU(2) and it acts on the normal space to Ki{p2) with respect to R-*^^ 
by fixing a 2-plane and by the vector representation on the orthogonal complement. So M 
has an element of the form 

/aOOOOOOX 
(7.9) 6 0, 

yOOcOdOO/ 
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for some c > 0. We can assume a > 6 by multiplying on the left by 



and on the right by 









/o 


1 














o\ 


1 


























-1 





























-1 

















-1 

















-1 











\0 

















1 / 



if necessary. Multiplying (|7.9| ) on the left by 



and on the right by 



we get 



/ 1 


















































1 







+ rf2 








be 






























d 



Vc^+d^ 










\ 






bd 




We can therefore assume b > a/c^ + (P. It is straightforward to check that no further restric- 
tions can be put on a, b, c, d. Hence we see that the orbit space is described by the matrices 
of the form ([r.9|) subject to the conditions a > b > vc? + tP^ and c > 0. 



Next we want to compute a principal isotropy subgroup of G. Let p be an element of the 
form ( [7.9|) which is regular for both G and G. The principal isotropy group Gp is isomorphic 



to Zl X SO (4), and Gp 



GpHG. 



Now the Z^-factor is contained in Gp, because 



(diag(-l, -1, 1), diag(-l, -1, 1, -1, 1, -1, 1)) 
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and 



(diag(l, -1, -1), diag(l, -1, -1, 1, -1, -1, 1)) 

are elements of G that fix p. On the other hand, an element in the SO(4)-factor that fixes 
p and is in G = SO(3) x G2 is in fact in {1} x G2 and has to fix e, i, cj + d{ei) G R^. 
Therefore it also fixes j = ^[{cj + d{ei)) — d{ei)] (c 7^ because p is regular for G) and hence 
it is the identity. We conclude that the principal isotropy group H = Gp is exactly Zg. 
It is immediate to see that the fixed subspace is 

* * 
* *0 
0*0*00 



Let be the normalizer of if in G and G = N/H the reduced group as in Subsection |373 . 
The connected component G° must be contained in {1} x G2. One can use the fact that 
N equals the normalizer of in G to verify that is the 2-torus group consisting of the 
matrices 



(7.10) 



/ cost 










— sint 










0\ 





cos s 















— sin s 











cos(s — 


t) 





— sin(s - 


-t) 








sint 










cost 



















sin(s — 


t) 





cos(s — 


t) 











sins 















cos s 





V 
























for s, t G R. 

Let p G be a regular point and let M = Gp be the corresponding principal orbit in 
V. We next prove that M is not taut by contradiction. In fact, assume M is taut. The orbit 
G^p is a connected component of , and it is a substantial 2-torus embedded in the 6- 
dimensional Euclidean space . Since its second osculating space is at most 5-dimensional, 
it cannot be taut by Theorem |2.5| . It is known that a surface is taut if and only if its lines 
of curvature are circles. Therefore there is a parallel normal vector field C,{t) along a curve 
7(t) in G^p C such that A^'^^^ •yit) = X(t)'j{t) and X{t) is not constant. (^4 posteriori it 
is clear that one can find 7(t), ^(t), X{t) directly by computation using (|7.10| ).) Since M is 
a principal G-orbit, the normal spaces of M and in V coincide along 7(t) implying that 
^ (t) is also parallel with respect to M. By Lemma p.l6| we have 



(7.11) 



Atlt)i{t) = Htm)- 



We do not claim that 7(t) is a curvature line of M since the eigenspace corresponding to A(t) 
might not be one-dimensional. 



Lemmas 1 and 2 (see also [pO 



Still the argument in p6 
carries through and shows that ( [7. Ill ) together with the tautness of M implies that A(t) is 
constant, which is a contradiction. Hence M is not taut. Thus the representation p is not 
taut either. □ 



7.12 Proposition The representations listed in Table C.l are taut. 
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Proof. We use the reduction technique discussed in Subsection pT3| . Consider first the 
case of G = U(2) x Sp(n), n > 2, acting on the reahfication of (S> C^" by the tensor 
product of the vector representations. It is equivalent to consider p acting on the space of 
2 X 2n complex matrices, i. e. p{A,B)X = AXB'^ for A e U(2), B e Sp(n) C SU(2n), 
X G M(2 X 2n; C). It follows rather easily that the orbit space consists of matrices of the 
form 

'aOO...OOOO...O 
6 ... c ... 



where a > y/b'^ + and b, c > 0. Since these matrices are all real, it follows that the group 
obtained by adjoining to G the complex conjugation of matrices a still has the same orbits. 
We replace G by the enlarged group and continue to denote it with the same letter. 

Now the principal isotropy group H is generated by a and the subgroup of G isomorphic 
to U(l) X Sp{n — 2) consisting of elements of the form 



A9 







\ 



-id 



V 



c 



-D 







A9 



D 



\ 



J 



where ^ G R and 



C 
-D 



D 
C 



e Sp{n - 2). Now is the space of matrices of the form 



a 
6 




c 



G {1} X Sp(n) 



whose entries are real, and (N/H)^ is the circle group consisting of the coclasses defined by 
the matrices 

/ cose sin^ \ 

cos^ sin 6' 

1 

— sin 6 cos 9 

sin 6* cos 6' 

V 1/ 

for 6^ G R, where 1 (resp. 0) denotes the (n — 2) x (n — 2) identity (resp. zero) block. We see 
that the reduced representation p : (N/H)^ — > 0(y^) is just the direct sum of two copies 
of the vector representation of SO(2) on R^, hence taut. Finally, we conclude that p is taut 
by Proposition |3.19| , where we choose L to be generated by the following elements: a, the 
diagonal elements in Sp(?T, — 2) (Z H with ibl entries, and the element 



i 
-i 



—I 



1 / 
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Next consider the case of G = SO(2) x Spin(9) acting on R^®R^^ by the tensor product 
of the vector and spin representations. It is equivalent to consider p acting on the space of 
2 X 16 real matrices, namely, p{A,B)X = AXB-^ for A e SO(2), B G Spin(9) C SO(16), 
X e M(2 X 16; R). We identify R^^ = O © O and use {1, e, ij, k, ei, ej, ek} as a basis of O 
over R. For the computations involved in this case. Lemmas 14.61 and 14.77 from are 
useful, which we state as follows: 

7.13 Lemma A matrix g G 0(8) = 0(0) belongs to Spin(7) if and only if g{uv) = 
g{u)x{g){v) for all u, v E O, where x{g){v) = g{g^^{l)v) for t; G O defines the vector 
representation o/Spin(7) on = R''. 



7.14 Lemma The subgroup Spin(9) C SO (16) = S0(0 © O) is generated by the 8-sphere 

I ^" V^eR,MGO,r2 + |up = 
Ru -r J 

in End(0 © O), where Ru ■ O ^ O denotes right translation by the element u E O. 
Mainly as a consequence of 

11 1 
one can see that the G-orbit space consists of matrices of the form 



(7.16) 



aOOOOOOOOOOOOOOO 
OfeOOOOOOcOOOOOOO 



and, with some more effort (using Lemma [7.14| ), that the restrictions a > yP + c? and b, 
c > apply. Also, Lemma [7.13| shows that 

gi = diag(-l, 1, 1, 1, -1 - 1 - 1, 1) G Spin(7), 

which combined with (|7.15|) yields that 

^2=1^ Xi9i) j e Spin(9), 

where xi9i) = diag(— 1, —1, —1, 1, 1, 1, —1). In particular, the equality ^ ^ P ~ 

where p is as in ( fTTel) shows that 0(2) x Spin(9) has the same orbits as SO(2) x Spin(9). 



Hereafter we consider G = 0(2) x Spin (9). 

The principal isotropy group H is isomorphic to Z2 x Z2 x SU(3), where the first Z2 
corresponds to the kernel of p and it is generated by the element hi = (—1, —1) G 0(2) x 
Spin (9), the second Z2 is generated by the element 

/i2 = (diag(l, -1),5'2), 
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and the SU(3)-factor consists of matrices of the form 



/ 1 



A B 
-B A 



A B 
-B A J 



E {1} X Spin(9), 



where A + iB E SU(3). Now V is the space of matrices of the form 

aOOOOOOOOrfOOOOOO 
06000000cOOOOOOO 

whose entries are real, and (N/H)^ is the circle group consisting of the coclasses defined by 
the following matrices of {1} x Spin (9): 



/ cos 9 



(7.17) 



sin 6 



\ 



cos 9 



sin 9 



cos 6*1 



cos6'l 



sin 6* 



cos 9 



— sin 91 



sin 91 



— sin 9 



cos 9 



— sin 91 



cos^^l 



V 



sin 91 



cos^l / 



for 9 E H, where 1 denotes the 3x3 identity block (note that the matrix (|7.17|) belongs 
to Spin(9) because it is the product of the two matrices in Spin(9) which are obtained by 
setting r = cos 6', u = sin6'e and r = 1, m = in Lemma |7. 14|) . We see that the reduced 
representation p is just the direct sum of two copies of the vector representation of SO(2) 
on R^, hence taut. We conclude that p is taut by Proposition |3.19| , where we choose L to 
be generated by hi, h2 and the elements in the SU(3)-factor of H with A diagonal with ±1 
entries and B = 0. 

Finally we consider the case of G = SU(2) x Sp(n), n > 2, acting on a real form 
V of ® C^". It is equivalent to consider p acting on a subspace of 4 x 2n complex 
matrices, namely, piA,B)Z = 7]{A)ZB-^ for A E SU(2), B E Sp{n) C SU(2n), Z eV = 
X Y 



-Y X 



E M(4 X 2n; C) k where r/ : SU(2) Sp(2) C SU(4) is the 4-dimensional 



complex representation of SU(2) given by 



a P 
—/3 a 



( 0? 



aa^-2af3(3 V3a^(3 (3(3^-2aa(3 

2afP ) 
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Let 



Pi 



/ 1 





\ 







10 








0/ 



G V. 



Then the orbit G{pi) is (4n + l)-dimensional and its normal space intersects all orbits at pi. 
So, given an arbitrary orbit M, it contains a point of the form 



(7.18) 



/ a 

Xi 



V 






-m -m 











a 








-yn Xi X2 Xs 



\ 

Vn 





where Xj, ?/i G C and a > 0, for we can multiply on the left by the minus identity matrix 
to change the sign of a, if necessary, and remain in the orbit M. The isotropy subgroup 
Ki = Gp^ is isomorphic to S"^ x Sp(n — 1), where the circle factor consists of the elements 



v 



( 



\ 



,316* 



-3ie 



1// 



for G R, where 1 denotes the (n — 1) x (n — 1) identity block. By letting K\ act on the 
matrix ( [7.1^ ) we now see that the G-orbit space consists of real matrices of the form 

/aOO...OOOO...O\ 

c 6 ... ... 

... a ... 
\000...0c60...0/ 

(One can also check that some restrictions on a, h and c apply, but this is unimportant to 
us.) 

The principal isotropy group 



E 




C 



(a,/3) = ±(l,0),±(2,0),±(0,l),±(0,z) 



is isomorphic to Q x Sp(n — 2), where Q is quaternion group {±1, ±i, ±j, ±/c}. Now is 
the space of matrices of the form 

/arfO...OOOO...O\ 

c 6 ... ... 

... a ... 
\000...0c60...0/ 
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where the entries are real, and (N/H)^ is the circle group consisting of the coclasses defined 
by the matrices 



/ cos 6* 
sin 6 



V 



— sin^ 
cos 6' 



\ 



cos^^ 
sin 6' 



— sin 6 
cos 9 



e {1} X Sp{n) 



1 / 



We see that the reduced representation p is just the direct sum of two copies of the vector 
representation of SO(2) on R^, hence taut. We conclude that p is taut by Proposition |3.19| , 
where we choose L to be generated by Q and the diagonal elements in Sp(n — 2) G H with 
±1 entries. □ 



8 Other applications 

In this last section we collect some further applications of our methods. 

8.1 Theorem (Dadok) A polar representation of a compact connected Lie group is orbit 
equivalent to the isotropy representation of a symmetric space. 

Proof. Conlon's theorem (Theorem |2.3| ) says that a polar representation is variationally 
complete. Now use Theorem |6.4 D 

The next theorem, which is a special case of a result of Montgomery and Samelson on 
transitive homeomorphic actions on spheres ([0), see also [|], ^ by Borel, was used in 
Sections | and ||. 

8.2 Theorem The transitive effective linear actions of compact connected Lie groups G on 
the spheres S"^ are given by the following tables (in each case, the action is the obvious one): 



G 


SO{n) 


G2 


Spin(7) 


Spin(9) 


Sp(i) • SpH 












^4n— 1 



G 


SU(n) 


■ SU(n) 


Sp(n) 


■ Sp{n) 


5^ 







Proof. We may assume that the action is given by a real irreducible representation p of 
G. It is enough to classify p up to image equivalence and then factor the kernel out. If p 
is of quaternionic or complex type, then Lemma |5.12| implies that k{X) = 1 and we get the 
standard action of SO (2) on S*^ and the actions in the second table. If p is of real type, then 
Lemmas |5.11| , |5.14| and ^.15| together give the remaining actions in the first table. □ 

Representations p : G ^ O(^) with cohomogeneity equal to two were classified up 
to orbit equivalence in |^ by Hsiang and LawsonQ. It is not difficult to prove that a 



^The classification of coliomogeneity two and tliree representations in [26| contain gaps which were to our 
knowledge first corrected in the papers Bil Efl] . 
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representation with cohomogeneity two is polar. Hence it is also taut and if it is irreducible 

it is therefore in class O^. 

We next show that it is quite easy to see directly that an irreducible representation with 
cohomogeneity two is in class O^. We formulate this in the next proposition. 

8.3 Proposition An irreducible representation p : G ^ O(^) ^^^^ cohomogeneity two is in 
class O'^. 

Proof. We can clearly restrict our attention to orbits contained in the unit sphere S 
of V . Let p in S* be a point whose orbit has codimension two. The second osculating space 
OpiGp) is the direct sum of Tp{Gp) and the linear span of {a{X, Y) \ X,Y G Tp{Gp)} where 
a is the second fundamental form of the orbit Gp as a submanifold of V. It is a general 
property of submanifolds contained in the sphere S that {a{X, X), ^) = 1 for all unit vectors 
X e Tp(Gp), where ^ is the inward pointing unit normal vector of S in p. We therefore need 
to prove that the linear span of {a{X,Y) \X,Y E Tp{Gp)} is not one dimensional. Let f] 
be a normal vector of Gp at p that is tangent to the sphere and let be the Weingarten 
map of Gp at p as a submanifold of S. The orbit Gp cannot be umbilical in S since then it 
would be a round sphere contradicting the assumption that p is irreducible. Hence A^ has 
at least two different eigenvalues Ai and A2 with corresponding eigenvectors Xi and X2 that 
we assume to have length one. Then 

a{Xi,Xi) = {a{Xi,Xi),OC+{a{Xi,Xi),rj)rj 

for i = 1,2. It follows that a{Xi,Xi) and a{X2,X2) are not linearly dependent and hence 
that V = OliGp). 

Now assume that the orbit through p in 5" is singular. If we restrict the action of p to 5", 
then we have a cohomogeneity one action. It is well known that the slice representations of 
cohomogeneity one actions are irreducible and transitive on spheres in the normal space. If 
we now think of the orbit Gp as a submanifold of V , then we see that the slice representation 
of Gp on the normal space Np{Gp) decomposes into the trivial representation on and an 
irreducible representation on N^{Gp) where ^ is a normal vector of the sphere S as above 
and Np{Gp) is the normal space of Gp in S. The linear span of {a{X, Y)\ X,Y e Tp{Gp)} is 
clearly invariant under the isotropy representation of Gp. It follows that it contains N.^{Gp) 
since it must have a component in Np{Gp). (If it had no component in Np{Gp), then Gp 
would be totally geodesic in S, hence a round sphere, contradicting the assumption that p 
is irreducible.) The hnear span of {a{X, Y)\X,Y e Tp{Gp)} cannot coincide with iV^(Gp), 
since a{X, X) ^ N^{Gp) for a nonvanishing X in Tp{Gp). It follows that V = Ol{Gp). □ 

8.4 Theorem (Hsiang-Lawson) The cohomogeneity two linear actions of compact con- 
nected Lie groups on the Euclidean spaces are given, up to orbit equivalence, by the isotropy 
representations of the rank two symmetric spaces. 

Proof. A cohomogeneity two reducible representation is the direct sum of two cohomo- 
geneity one representations and therefore orbit equivalent to the isotropy representation of 
the product of two spheres. 
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A cohomogeneity two irreducible representation of a compact connected Lie group is of 
class (Proposition |8.3| ). Therefore the result follows from the fact that no representation 
listed in Tables C.l, C.2, C.3 and C.4 is of cohomogeneity two, which is easy to check. □ 



A Description of the maximal subsets of strongly or- 
thogonal roots of the complex simple Lie algebras 

In this appendix, for each one of the complex simple Lie algebras, we give a description of 
the maximal subset of strongly orthogonal roots B = . . . which was introduced in 



Proposition |4T|. In each case, A+ is the positive root system, S = {ai, . . . , «„} is the simple 
root system, and Ai, . . . , A.^ are the fundamental highest weights {9i, 62, . . . refer to specific 
linear forms on the Cartan subalgebra; see e. g. 0). 



ai OL-2 

A.YI : 


an 

— 










S={ai=9i- 


- ^2, • • • , 










A+ = {0, - 9, 


l<i<j<n + l} 








A2 = ^1 + 




Pi = 01 — dn+1 
P2 = 02 — dn 


Pi 
P2 


= 0i- 
= 02- 




= Oi + 02 + 




Pn = 0^ - 0^+2 

{n even) 


Pn 

{n 


= 6 n + 1 

odd)' 


— On + l,-, 
2 ' 





ai 0.2 




-1 an 






















S = 


= {ai = 9i - 


-02, 


■ ■ , Oin-1 — dn-1 — On, O-n — 6'n} 






A+ 


= {9^ ± 9j 


■ I < i < j < n} U {Oi : 1 < n} 






Ai 


= 0i 






Pl = Oi + O2 


Pi 


= 0i+02 


A2 


= 9^+02 






P2 = Oi — O2 


P2 


= 61 — 62 


An- 


-2 = ^1 + 62 




• + ^n-2 


Pn-2 = On-2 + On-1 


Pn 


-1 = 6n-l + 6n 


An- 


-1 = dl + 02 




• + On-2 + On-1 


Pn-1 = On-2 — On-1 


Pn 


= 6n-l — 6n 


An 


= ^((^1+02 




■ + On-1 + On) 


Pn = On 














{n odd) 


{n 


even) 





ai Q2 


Ctn — 1 n 




Cn 


: • • — ■■• 






s = 


= {ai = 61- 


62,- ■ ■ , 


ttn-l = 6n-l — 6n, an = 26n} 


A+ 


= {6i ± 6j : 


1 < z < j < n} U {26i : 1 < n} 


Ai 


= 61 




Pi = 26i 


A2 


= 61 + 62 




P2 = 262 


An 


= 61 + 62 + 


■ ■ + 6n 


Pn = 26n 
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S = 


= -[q;i = 6^ — ^9, . . • 1 Oir,—'? — On— 9 


— Uri—A 1 Oiri—A Ur^ — ^ ~ 

11 — ±7 ^^11 — ± 111 — X 


in 1 i It 


-1 + 9n} 


A+ 


= {^i ± : 1< i < J < n} 








Ai 




Pi = 9^ + 02 


Pl = 01 + 02 




A2 


= 01+92 


P2 = 9i — 02 


^2 = 01 — 02 




An- 


-2 = 9i + 92 + . ■ ■ + 9n-2 


(3n-2 = 9n-2 + 


Pn-l = 9,1-1 


+ 9n 


An- 


-1 = 1(^1 + ^2 + • • • + 9n-l — 9n) 


/^n-l = 9n-2 — 9n-l 
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An 


= \i9l + 92 + . . . + 9n-i + 9n) 
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{n even) 
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j_ 02 
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= {ai, a2, ai + a2, 2ai + a2, 3q;i + a2, 3q;i + 2a2} 
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= 2«i + a2 
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(52 = OLl 
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<x\ a2 0:3 ct4 






S -- 


= {«! = 02 - 03, a2 = 03 


— ^4, 


a3 = 04,a, = i{0i- 02 - 03 - 0A)} 


A-+ 


- = {9i:l<i<4}U{9i± 9j 


: 1 < z < j < 4} U {^(^1 ± ^2 ± ^3 ± ^4)} 


Ai 


= 9i + 92 


Pl = 


01 + 92 


A2 


= 29i + ^2 + ^3 


P2 = 


9i — 92 


A3 
A4 


= 1(3^1+^2 + ^3 + ^4) 
= 01 


P3 = 
P4 = 


93 + 9a 

■03 — 04, 



ai 03 a4 05 OCQ 

Ee : ^ 


S = {ai = ^{i 
a2 = 01 + 


h — 92 — 9^ — 9, — 9^ — 9(i — 9i + 9s), 
92, Oi3 = 92 — 9i, a, = 9^ — 92, cts = 6*4 — 9^, = 6*5 — 6*4} 


A+ = {±9i + 


: 1 < i < j < 5} U {\{0s -07-06 + E-=l ^^0^) ■ ^Ll^i = +1} 


Ai = |(^8 — 9r — 9e) 

A2 = 2(^1 + ^2 + ^3 + ^4 + ^5 — ^6 — ^7 + ^s) 
A3 = K^^S - ^^7 - 9e) + + ^2 + ^3 + ^4 + ^5) 
A4 = ^^3 + ^^4 + ^^5 — ^^6 — ^7 + ^8 
A5 = j{98 — 9y — 9q) + 6*4 + 6*5 

Ae = 3(^8 — 07 — 0&) + 0b 


/9l = i(^l+^2 + ^3 + ^4 

+05 — 06 — 07 + 08) 
(32 = \{,—01 — 02 — 03 — 9i 
+^5 — ^6 — 6*7 + ^8) 

(3^ = 9i — 9i 

^4 = 03 — 02 
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CKi 03 04 05 ag CK7 


















c 

o = 


{«i = \{0i — 92 — 0^ — 64 — 9^ — 9q — 9-j + 9s), 








a2 = 9i + 92, aa = 6*2 — 6^1, q;4 = ^3 — ^2, "5 = 


04 


- t>s, ae — (75 — 


(74, a7 — — (75]- 




= {±9i + 9r.l<i<j <6}lJ{0s- 07} U {i(^8 


/I 1 \ — ^t) / 

-^7 + Ei=l ^i^ 


%) ■■ ntiQ = -1} 


Ai = 


- 9s — 97 


Pi 


n Q 
— Us — 07 




A2 = 


\{9^ + 92 + 9^ + 9^ + 9^ + 9^- 29j + 29s) 




= 06 + 0b 




A3 = 


li-9, + 92 + 9s + 9^ + 9^ + 9^- 397 + 39s) 




= 06 — 05 




A4 = 


--93 + 94 + 95 + 9e + 2{9s - ^7) 




= 04 + 03 




A5 = 


= 1(2^4 + 2^5 + 2^6 + 3(^8-^7)) 




= 04 — 03 




A6 = 


- 05 + 9q — 97 + 9s 


P6 


= 02 + 01 




A7 = 


- 06 + ^{08 — 07) 




= 02 — 01 







ai CK3 CK4 CK5 CKg CK7 CKg 






E8 : 








S = 


{«1 = 1(^1 - ^^2 - ^^3 - ^4 - ^5 - ^6 - ^7 + ^S), "2 


= 01+ 02, 




0:3 = ^2 - ^1, 0:4 = ^3 - ^2, a.^ = 04 - 03, tte = 


= 05- 


04, a7 = 0e — 05, as = 07 — ^e} 


A+ 


= {±e, + e,:l<i<i<8}U{i(e8 + ELi 


^10 i) 


nJ^iQ = +1} 


Ai = 


--20S 


Pl = 


07 + 08 


A2 = 


-- ^{01 + ^2 + ^3 + ^4 + ^5 + ^6 + ^7 + 5^8) 


^2 = 


08 — 07 


A3 = 


- + ^2 + ^3 + ^^4 + ^^5 + ^^6 + ^7 + r^s) 


^3 = 


06 + 05 


A4 = 


= ^3 + ^4 + ^5 + ^6 + ^^7 + 5^8 


Pa = 


06 — 05 


A5 = 


= ^4 + ^5 + ^6 + ^7 + 4.0s 


P5 = 


04 + 03 


A6 = 


- 05 + ^6 + ^7 + 3^8 


P6 = 


04 — 03 


A7 = 


- 0Q + 07 + 20s 


^7 = 


02 + 01 


A8 = 


- 07 + 0s 


Ps = 


02 — 01 



B Table for Dadok's invariant k{X) 

We next tabulate the value of the Dadok invariant k{X) for the the complex irreducible 
representations tta of the complex simple Lie algebras, where A is a fundamental highest 
weight (since A;(A) is linear on A, this already determines the value of A;(A) for all A). The 
number in parenthesis next to the vertex of the Dynkin diagram which specifies ctj is k{Xi). 
The last column tells, for each complex simple Lie algebra, whether all representations are 
self-dual, or else, the dual of tta^ is given by ttx^ , where the vertices that specify ai and aj 
are images one of the other under the symmetry of the Dynkin diagram. 
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G 


MA) 


Conditions 


A2n-1 


(1) (2) (n-l) (n) (n-l) (2) (1) 


duality given by sym- 
metry of diagram 




(1) (2) („-l) (n) (n) (n-l) (2) (1) 


duality given by sym- 
metry of diagram 


B2n-1 


(2) (2) (4) (4) (2n - 2)(2n - 2) (n) 


all self- dual 


B2n 


(2) (2) (4) (4) (2n - 2X2n - 2)(2n) (n) 


all self-dual 


Cn 


(1) (2) (n-l) (n) 

• • ^=0 


all self-dual 


D2n 


(2) (2) (4) (4) (2n - 2)(2n - 2)P ^"'> 

D (n) 


all self-dual 


D2n+1 


(2) (2) (4) (4) (2n - 2)(2n - 2X2n)P 

(n) 


duality given by sym- 
metry of diagram 


G2 


(2) (2) 


all self-dual 


F4 


(2) (6) (4) (2) 
(2) (4) (6) (4) (2) 


all self-dual 


Ee 


(2) (6) (8) (7) (4) (3) 


duality given by sym- 
metry of diagram 


Ey 


(4) (10) (14) (12) (8) (6) (2) 


all self-dual 


Eg 




all self-dual 



Table B.l: The invariant of Dadok. 
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C Table of candidates to class 



In the tables below are compiled all the real irreducible representations of compact connected 
Lie groups that are not orbit equivalent to the isotropy representation of a symmetric space 
but that have a chance of being a representation of class C^. These are just the representa- 
tions mentioned in Propositions ^TT| , |5.17| , p.l8| , ^.19| , ^.2U| and |5.21| , except those excluded 
by the discussion in Section ^ note however that the presentation here slightly differs from 
the presentation in the above mentioned propositions for the sake of convenience. 



G 


P 


Conditions 


SO(2) X Spin(9) 
U(2) X Sp{n) 
SU(2) X Sp(n) 


1 

1 1 

3 1 


n>2 
n>2 



Table C.l: Taut representations. 
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G 



Conditions 



SU(n) X Sp(m) 
U(n) X Sp(m) 



SO(m) X Sp(l) • Sp(n) 

{n > 2) 



SO(m) X G2 



SO(m) X Spin(7) 



SO(m) X Spin(9) 



(o O e O o) 



(X 



— o® 

1 



-o) 



g)0( 

1 1 

O (g) O ( 

1111 

O (g) O (g) O (g) •- 

2 11 

0(g)0(g)«— ■ 



1 1 

111 
0(g 0(g 

2 1 



0(g)^^D 



1 



— o=« I 



1 



0(g)0(g)0 — c^=« 




n > 3, m > 2 
n > 3, m > 2 



m = 2p > 4 
m = 2p + 1 > 3 
m = 4 
m = 3 



m = 2p > 4 
m = 2p + 1 > 3 
m = 4 

m = 3 



m = 2p > 4 
m = 2p + 1 > 5 
m — 4 



m — 2p > 4 
m = 2p + 1 > 3 
m = 4 
m = 3 



Table C.2: Nontaut representations (first group). 



84 



G 



Conditions 



G2 X G2 

G2 X Spin(7) 
G2 X Spin(9) 
Spin(7) X Spin(7) 
Spin(7) X Spin(9) 
Sp(l) • Sp(n) X G2 
Sp(l) ■ Sp(n) X Spin(7) 




1 



1 1 

1 1 



1 

=• 



n>2 

n> 2 



Table C.3: Nontaut representations (second group). 



G 


P 


Conditions 


Sp(n) X SU(6) 


1 1 


n>2 


Sp(n) X Spin(ll) 


1 1 




Sp(n) X Spin(12) 


• 9=0 (8) 


n>2 


Sp(l) X Spin(13) 


1 1 




Sp(l) X Sp(2) 


111 




Sp(n) X Sp(3) 


1 1 
• •=0 (g) • •=0 

1 


n>2 


Sp(n) X E7 


• •=o (g) ° ° ^ ° ° ° 


n>2 


Spin(7) 


1 1 




Spin(9) 


1 1 




Spin(15) 


1 




Spin(17) 


1 





Table C.4: Nontaut representations (third group). 
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